{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,11,5]],"date-time":"2026-11-05T22:29:22Z","timestamp":1793917762203,"version":"4.8.0"},"reference-count":23,"publisher":"Oxford University Press (OUP)","license":[{"start":{"date-parts":[[2026,9,22]],"date-time":"2026-09-22T00:00:00Z","timestamp":1790035200000},"content-version":"am","delay-in-days":0,"URL":"https:\/\/academic.oup.com\/pages\/standard-publication-reuse-rights"}],"funder":[{"name":"Ratchadapiseksompotch Fund","award":["RA-MF-23\/63"],"award-info":[{"award-number":["RA-MF-23\/63"]}]},{"name":"Ratchadapiseksompotch Fund","award":["RES_63_028_30_006"],"award-info":[{"award-number":["RES_63_028_30_006"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:sec>\n                    <jats:title>Purpose<\/jats:title>\n                    <jats:p>This national study examined longitudinal changes in perceived feedback quality during phased rollout of a mobile entrustable professional activity-based feedback application across Thai College of Emergency Physicians-accredited emergency medicine residency programs and explored institutional, contextual, and application use variables associated with perceived feedback quality.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Method<\/jats:title>\n                    <jats:p>This national, multisite prospective observational study (December 2019-September 2020) included 135 faculty and 357 residents from accredited Thai emergency medicine residency programs. The application was implemented in 3 phases: notifications, feedback guide\/self-reflection prompts, and dashboard. Participants completed Feedback in the Emergency Room across 5 domains (timeliness, specificity, communication, action plan, feedback culture) using 5-point Likert-type items. Scores were converted to Percent of Maximum Possible (POMP; 0-100; higher scores indicate higher perceived feedback quality). Linear mixed-effects models estimated changes in domain and total Feedback in the Emergency Room POMP scores over time; total score models assessed associations with institutional, contextual, and application use variables.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>Faculty-reported overall perceived feedback quality did not change significantly at 3 months but increased at 6 months (4.03 POMP; 95% CI, 1.99-6.07) and 9 months (3.57; 95% CI, 1.02-6.11). Resident-reported overall scores did not change significantly. In resident models, dashboard review (always: 2.54; 95% CI, 0.18-4.89; sometimes: 2.21; 95% CI, 0.32-4.09), and resident rules (1.78; 95% CI, 0.09-3.48) were associated with higher scores; absence of formal feedback strategies was associated with lower scores (\u22122.00; 95% CI, \u22123.87 to \u22120.13).<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Conclusions<\/jats:title>\n                    <jats:p>During implementation of a mobile entrustable professional activity-based feedback application, faculty reported modest improvements in perceived feedback quality, whereas resident perceptions remained stable. Digital tools may support feedback processes when aligned with institutional context. Future studies should link feedback to encounters and evaluate mechanisms and downstream educational outcomes using designs that strengthen causal inference.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.1093\/acamed\/wvag290","type":"journal-article","created":{"date-parts":[[2026,9,22]],"date-time":"2026-09-22T13:58:31Z","timestamp":1790085511000},"source":"Crossref","is-referenced-by-count":0,"title":["Longitudinal changes in feedback quality after nationwide implementation of a mobile application for entrustable professional activity data management in Thailand"],"prefix":"10.1093","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-5191-6153","authenticated-orcid":false,"given":"Khuansiri","family":"Narajeenron","sequence":"first","affiliation":[{"name":"Department of Emergency Medicine, Faculty of Medicine, Chulalongkorn University and King Chulalongkorn Memorial Hospital , Bangkok,","place":["Thailand"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1919-4525","authenticated-orcid":false,"given":"Stephen J","family":"Kerr","sequence":"additional","affiliation":[{"name":"Biostatistics Excellence Centre, Faculty of Medicine, Chulalongkorn University , Bangkok,","place":["Thailand"]},{"name":"The Kirby Institute, University of New South Wales , Sydney, New South Wales,","place":["Australia"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8748-9360","authenticated-orcid":false,"given":"Intanon","family":"Imsuwan","sequence":"additional","affiliation":[{"name":"Department of Emergency Medicine, Faculty of Medicine, Thammasat University , Pathum Thani,","place":["Thailand"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2750-3981","authenticated-orcid":false,"given":"Cherdsak","family":"Iramaneerat","sequence":"additional","affiliation":[{"name":"Department of Surgery and Siriraj Health Science Education Excellence Center, Faculty of Medicine Siriraj Hospital, Mahidol University , Bangkok,","place":["Thailand"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9252-1588","authenticated-orcid":false,"given":"Ara","family":"Tekian","sequence":"additional","affiliation":[{"name":"Department of Medical Education, University of Illinois College of Medicine , Chicago, IL,","place":["United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8583-4335","authenticated-orcid":false,"given":"Yoon Soo","family":"Park","sequence":"additional","affiliation":[{"name":"Department of Medical Education, University of Illinois College of Medicine , Chicago, IL,","place":["United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"name":"TACTIC Study Group","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Khrongwong","family":"Musikatavorn","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Tanawat","family":"Tarapan","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Natchaya","family":"Treesaksrisakul","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Supattra","family":"Inyu","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ar-aishah","family":"Dadeh","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Suthee","family":"Intharachat","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Piyatida","family":"Kalayanamitra","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Pornlert","family":"Pluemchitmongkhon","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Panuwong","family":"Sansomranjai","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Suwasan","family":"Bunyarat","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Marisa","family":"Thongnok","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sutjarit","family":"Suankool","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Chanipat","family":"Meelarp","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Kamonphan","family":"Thirapong","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Chompunut","family":"Saengpanit","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Narong","family":"Jongaramrueng","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Napat","family":"Rattanawongsa","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Chaiyaporn","family":"Yuksen","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Chanida","family":"Kemngern","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Kannika","family":"Tasasungkhin","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Theerapon","family":"Tangsuwanaruk","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Varalee","family":"Aphinives","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Nutarpa","family":"Kulkanokwan","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Phummarin","family":"Saelim","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Dhanu","family":"Gaysonsiri","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Watchara","family":"Petchratchatanon","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Alisa","family":"Yanasan","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Tanyaporn","family":"Nakornchai","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Suttisak","family":"Wattanasansomboon","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"286","published-online":{"date-parts":[[2026,9,22]]},"reference":[{"key":"2026110517171537200_wvag290-B1","doi-asserted-by":"publisher","first-page":"676","DOI":"10.3109\/0142159X.2010.500704","article-title":"The role of assessment in competency-based medical education","volume":"32","author":"Holmboe","year":"2010","journal-title":"Med Teach"},{"key":"2026110517171537200_wvag290-B2","doi-asserted-by":"publisher","first-page":"983","DOI":"10.3109\/0142159X.2015.1060308","article-title":"Curriculum development for the workplace using Entrustable Professional Activities (EPAs): AMEE guide no. 99","volume":"37","author":"Ten Cate","year":"2015","journal-title":"Med Teach"},{"key":"2026110517171537200_wvag290-B3","doi-asserted-by":"publisher","first-page":"701","DOI":"10.1080\/0142159X.2023.2197137","article-title":"Twelve tips to develop entrustable professional activities","volume":"45","author":"Hennus","year":"2023","journal-title":"Med Teach"},{"key":"2026110517171537200_wvag290-B4","doi-asserted-by":"publisher","first-page":"787","DOI":"10.3109\/0142159X.2012.684916","article-title":"Twelve tips for giving feedback effectively in the clinical environment","volume":"34","author":"Ramani","year":"2012","journal-title":"Med Teach"},{"key":"2026110517171537200_wvag290-B5","doi-asserted-by":"publisher","first-page":"1342","DOI":"10.1080\/0142159X.2019.1656804","article-title":"Meaningful feedback through a sociocultural lens","volume":"41","author":"Ramani","year":"2019","journal-title":"Med Teach"},{"key":"2026110517171537200_wvag290-B6","doi-asserted-by":"publisher","first-page":"692","DOI":"10.3109\/0142159X.2014.889812","article-title":"Unfulfilled promise, untapped potential: feedback at the crossroads","volume":"36","author":"Watling","year":"2014","journal-title":"Med Teach"},{"key":"2026110517171537200_wvag290-B7","doi-asserted-by":"publisher","first-page":"e32","DOI":"10.36834\/cmej.69913","article-title":"Feedback on feedback: a two-way street between residents and preceptors","volume":"12","author":"Griffiths","year":"2021","journal-title":"Can Med Educ J"},{"key":"2026110517171537200_wvag290-B8","doi-asserted-by":"publisher","first-page":"609","DOI":"10.1097\/ACM.0000000000000560","article-title":"The \u201ceducational alliance\u201d as a framework for reconceptualizing feedback in medical education","volume":"90","author":"Telio","year":"2015","journal-title":"Acad Med"},{"key":"2026110517171537200_wvag290-B9","doi-asserted-by":"publisher","first-page":"192","DOI":"10.1111\/j.1365-2923.2011.04126.x","article-title":"Learning from clinical work: the roles of learning cues and credibility judgements","volume":"46","author":"Watling","year":"2012","journal-title":"Med Educ"},{"key":"2026110517171537200_wvag290-B10","doi-asserted-by":"publisher","first-page":"60","DOI":"10.1111\/j.1553-2712.2010.00964.x","article-title":"Direct observation of clinical practice in emergency medicine education","volume":"18","author":"Craig","year":"2011","journal-title":"Acad Emerg Med"},{"key":"2026110517171537200_wvag290-B11","doi-asserted-by":"publisher","first-page":"479","DOI":"10.5811\/westjem.56544","article-title":"Feedback in medical education: an evidence-based guide to best practices from the Council of Residency Directors in Emergency Medicine","volume":"24","author":"Natesan","year":"2023","journal-title":"West J Emerg Med"},{"key":"2026110517171537200_wvag290-B12","doi-asserted-by":"publisher","DOI":"10.1111\/j.1553-2712.2009.00592.x","article-title":"Attending and resident satisfaction with feedback in the emergency department","volume":"16","author":"Yarris","year":"2009","journal-title":"Acad Emerg Med"},{"key":"2026110517171537200_wvag290-B13","doi-asserted-by":"publisher","first-page":"1261","DOI":"10.1097\/ACM.0000000000005305","article-title":"The assessment burden in competency-based medical education: how programs are adapting","volume":"98","author":"Szulewski","year":"2023","journal-title":"Acad Med"},{"key":"2026110517171537200_wvag290-B14","doi-asserted-by":"publisher","first-page":"35","DOI":"10.5811\/westjem.2018.11.38912","article-title":"Integration of entrustable professional activities with the milestones for emergency medicine residents","volume":"20","author":"Hart","year":"2019","journal-title":"West J Emerg Med"},{"key":"2026110517171537200_wvag290-B15","doi-asserted-by":"publisher","first-page":"1139","DOI":"10.1080\/0142159X.2021.1957088","article-title":"Ottawa 2020 consensus statement for programmatic assessment - 1. Agreement on the principles","volume":"43","author":"Heeneman","year":"2021","journal-title":"Med Teach"},{"key":"2026110517171537200_wvag290-B16","doi-asserted-by":"publisher","first-page":"555","DOI":"10.4300\/JGME-D-13-00409.1","article-title":"The milestones passport: a learner-centered application of the milestone framework to prompt real-time feedback in the emergency department","volume":"6","author":"Yarris","year":"2014","journal-title":"J Grad Med Educ"},{"key":"2026110517171537200_wvag290-B17","doi-asserted-by":"publisher","first-page":"e10849","DOI":"10.1002\/aet2.10849","article-title":"Examining enablers and barriers to entrustable professional activity acquisition using the theoretical domains framework: a qualitative framework analysis study","volume":"7","author":"Paterson","year":"2023","journal-title":"AEM Educ Train"},{"key":"2026110517171537200_wvag290-B18","doi-asserted-by":"publisher","first-page":"1203","DOI":"10.1080\/0142159X.2023.2168527","article-title":"Mobile technologies to support workplace-based assessment for entrustment decisions: guidelines for programs and educators: AMEE guide no. 154","volume":"45","author":"Marty","year":"2023","journal-title":"Med Teach"},{"key":"2026110517171537200_wvag290-B19","doi-asserted-by":"publisher","first-page":"14","DOI":"10.36834\/cmej.73554","article-title":"Developing a dashboard for program evaluation in competency-based training programs: a design-based research project","volume":"13","author":"Yilmaz","year":"2022","journal-title":"Can Med Educ J"},{"key":"2026110517171537200_wvag290-B20","doi-asserted-by":"publisher","first-page":"e10698","DOI":"10.1002\/aet2.10698","article-title":"Novel tool for assessing the quality of feedback in the emergency room (FEED-ER)","volume":"5","author":"Natesan","year":"2021","journal-title":"AEM Educ Train"},{"key":"2026110517171537200_wvag290-B21","doi-asserted-by":"publisher","first-page":"162","DOI":"10.1080\/10401334.2017.1392863","article-title":"Trainees\u2019 perceptions of feedback: validity evidence for two FEEDME (feedback in medical education) instruments","volume":"30","author":"Bing-You","year":"2018","journal-title":"Teach Learn Med"},{"key":"2026110517171537200_wvag290-B22","doi-asserted-by":"publisher","first-page":"315","DOI":"10.1207\/S15327906MBR3403_2","article-title":"The problem of units and the circumstance for POMP","volume":"34","author":"Cohen","year":"1999","journal-title":"Multivariate Behav Res"},{"key":"2026110517171537200_wvag290-B23","doi-asserted-by":"publisher","first-page":"267","DOI":"10.1016\/j.jclinepi.2013.08.015","article-title":"Systematic review of the Hawthorne effect: new concepts are needed to study research participation effects","volume":"67","author":"McCambridge","year":"2014","journal-title":"J Clin Epidemiol"}],"container-title":["Academic Medicine"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/academicmedicine\/advance-article-pdf\/doi\/10.1093\/acamed\/wvag290\/71313453\/wvag290.pdf","content-type":"application\/pdf","content-version":"am","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/academicmedicine\/advance-article-pdf\/doi\/10.1093\/acamed\/wvag290\/71313453\/wvag290.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,11,5]],"date-time":"2026-11-05T22:17:24Z","timestamp":1793917044000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/academicmedicine\/advance-article\/doi\/10.1093\/acamed\/wvag290\/8828065"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,9,22]]},"references-count":23,"URL":"https:\/\/doi.org\/10.1093\/acamed\/wvag290","relation":{},"ISSN":["1938-808X"],"issn-type":[{"value":"1938-808X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,9,22]]},"article-number":"wvag290"}}