{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,24]],"date-time":"2026-09-24T07:03:19Z","timestamp":1790233399420,"version":"4.1.0"},"reference-count":10,"publisher":"Wiley","issue":"5","license":[{"start":{"date-parts":[[2026,9,23]],"date-time":"2026-09-23T00:00:00Z","timestamp":1790121600000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"},{"start":{"date-parts":[[2026,9,23]],"date-time":"2026-09-23T00:00:00Z","timestamp":1790121600000},"content-version":"tdm","delay-in-days":0,"URL":"http:\/\/doi.wiley.com\/10.1002\/tdm_license_1.1"}],"content-domain":{"domain":["onlinelibrary.wiley.com"],"crossmark-restriction":true},"short-container-title":["AEM Education and Training"],"published-print":{"date-parts":[[2026,10]]},"abstract":"ABSTRACT<\/jats:title>\n \n Background<\/jats:title>\n Email reminders facilitate timely task completion but can create substantial administrative burdens for clerkship staff. In our emergency medicine (EM) clerkship, third\u2010year medical students initiate digital evaluation requests after each clinical shift, requiring frequent manual reminders to maintain completion rates. The time\u2010intensive nature of this process highlights the need for a more efficient and scalable approach.<\/jats:p>\n <\/jats:sec>\n \n Methods<\/jats:title>\n This multi\u2010site, pre\u2010post study evaluated clinical performance evaluation completion rates and time to completion during the required EM clerkship over 48\u2009weeks before and after implementation of an automated reminder system in September 2024. The system integrated existing survey functionality with Microsoft Excel and Microsoft Power Automate to track evaluation requests and completion and generate reminders every 2\u2009days for up to 28\u2009days. Primary outcomes were evaluation completion rate and time to completion. A theoretical cost analysis compared estimated personnel costs of the manual reminder process, based on administrator time and median salary estimates, with projected programming development and maintenance costs over 3\u2009years.<\/jats:p>\n <\/jats:sec>\n \n Results<\/jats:title>\n \n A total of 5747 evaluations were sent during this period to attending and resident evaluators. Overall completion rate rose modestly post\u2010intervention (71.4% to 75.1%,\n p<\/jats:italic>\n \u2009<\u20090.001), with attending evaluation rate demonstrating greater improvement from 73.9% to 81.6% (\n p<\/jats:italic>\n \u2009<\u20090.001). Conversely, both resident (median 2.67 to 3.98\u2009days) and attending completion time increased (median 2.59 to 2.74\u2009days),\n p<\/jats:italic>\n \u2009=\u20090.002. Based on the state salary estimates and our institutional experience, the automated approach could yield estimated net savings of $2850 in the first year and $9560 over 3\u2009years.\n <\/jats:p>\n <\/jats:sec>\n \n Conclusions<\/jats:title>\n Automated reminders improved evaluation completion but did not improve timeliness of completion, which may not translate into meaningful formative benefits if feedback is delayed. Further studies should focus on the impact on cognitive burden and evaluation quality before large\u2010scale automated systems can be optimized and sustainably integrated into medical education.<\/jats:p>\n <\/jats:sec>","DOI":"10.1002\/aet2.70257","type":"journal-article","created":{"date-parts":[[2026,9,24]],"date-time":"2026-09-24T06:45:07Z","timestamp":1790232307000},"update-policy":"https:\/\/doi.org\/10.1002\/crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Using an Automated Evaluation System to Improve Medical Student Evaluation Response Rate and Reduce Administrative Burden"],"prefix":"10.1002","volume":"10","author":[{"ORCID":"https:\/\/orcid.org\/0009-0009-8965-8207","authenticated-orcid":false,"given":"Chaiya","family":"Laoteppitaks","sequence":"first","affiliation":[{"name":"Department of Emergency Medicine Thomas Jefferson University Philadelphia Pennsylvania USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Alisha","family":"Agarwal","sequence":"additional","affiliation":[{"name":"Sidney Kimmel Medical College Philadelphia Pennsylvania USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0002-4572-8671","authenticated-orcid":false,"given":"Jason","family":"Gonzalez","sequence":"additional","affiliation":[{"name":"Department of Emergency Medicine Thomas Jefferson University Philadelphia Pennsylvania USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7350-8604","authenticated-orcid":false,"given":"Nina","family":"Mingioni","sequence":"additional","affiliation":[{"name":"Department of Internal Medicine Thomas Jefferson University Philadelphia Pennsylvania USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0103-9594","authenticated-orcid":false,"given":"Xiao Chi","family":"Zhang","sequence":"additional","affiliation":[{"name":"Department of Emergency Medicine Thomas Jefferson University Philadelphia Pennsylvania USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"311","published-online":{"date-parts":[[2026,9,23]]},"reference":[{"key":"e_1_2_12_2_1","doi-asserted-by":"publisher","DOI":"10.1186\/s12909\u2010024\u201005267\u20108"},{"key":"e_1_2_12_3_1","doi-asserted-by":"publisher","DOI":"10.3109\/0142159X.2014.886009"},{"key":"e_1_2_12_4_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.amjsurg.2017.08.047"},{"key":"e_1_2_12_5_1","doi-asserted-by":"publisher","DOI":"10.7326\/M16\u20100961"},{"key":"e_1_2_12_6_1","volume-title":"Research Brief: Administrative Waste's Role in Excess US Health Spending","year":"2022"},{"key":"e_1_2_12_7_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11606\u2010023\u201008153\u2010z"},{"key":"e_1_2_12_8_1","volume-title":"Emergency Medicine Clerkship Coordinators Handbook","author":"Byrne R.","year":"2025"},{"key":"e_1_2_12_9_1","doi-asserted-by":"publisher","DOI":"10.3390\/bioengineering11040337"},{"key":"e_1_2_12_10_1","volume-title":"Residency Program Coordinator Salary in Philadelphia, Pennsylvania","year":"2026"},{"key":"e_1_2_12_11_1","doi-asserted-by":"publisher","DOI":"10.5811\/westjem.59267"}],"container-title":["AEM Education and Training"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/pdf\/10.1002\/aet2.70257","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/full-xml\/10.1002\/aet2.70257","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/pdf\/10.1002\/aet2.70257","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,9,24]],"date-time":"2026-09-24T06:45:09Z","timestamp":1790232309000},"score":1,"resource":{"primary":{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/10.1002\/aet2.70257"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,9,23]]},"references-count":10,"journal-issue":{"issue":"5","published-print":{"date-parts":[[2026,10]]}},"alternative-id":["10.1002\/aet2.70257"],"URL":"https:\/\/doi.org\/10.1002\/aet2.70257","archive":["Portico"],"relation":{},"ISSN":["2472-5390","2472-5390"],"issn-type":[{"value":"2472-5390","type":"print"},{"value":"2472-5390","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,9,23]]},"assertion":[{"value":"2026-05-08","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2026-09-09","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2026-09-23","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}],"article-number":"e70257"}}