{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,8]],"date-time":"2026-02-08T05:05:25Z","timestamp":1770527125045,"version":"3.49.0"},"reference-count":30,"publisher":"MDPI AG","issue":"3","license":[{"start":{"date-parts":[[2021,2,24]],"date-time":"2021-02-24T00:00:00Z","timestamp":1614124800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Australian SIEF STEM+ class business fellowship in conjunction with the industry partner Cinglevue International Pty Ltd","award":["NA"],"award-info":[{"award-number":["NA"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Future Internet"],"abstract":"<jats:p>As COVID-19 continues to impact upon education worldwide, systems and organizations are rapidly transiting their professional learning to online mode. This raises concerns, not simply about whether online professional learning can result in equivalent outcomes to face-to-face learning, but more importantly about how to best evaluate online professional learning so we can iteratively improve our approaches. This case study analyses the evaluation of an online teacher professional development workshop for the purpose of critically reflecting upon the efficacy of workshop evaluation techniques. The evaluation approach was theoretically based in a synthesis of six seminal workshop evaluation models, and structured around eight critical dimensions of educational technology evaluation. The approach involving collection of pre-workshop participant background information, pre-\/post-teacher perceptions data, and post-workshop focus group perceptions, enabled the changes in teacher knowledge, skills, and beliefs to be objectively evaluated, at the same time as providing qualitative information to effectively improve future iterations of the workshops along a broad range of dimensions. The evaluation approach demonstrated that the professional learning that was shifted into online mode in response to COVID-19 could unequivocally result in significant improvements to professional learning outcomes. More importantly, the evaluation approach is critically contrasted with previous evaluation models, and a series of recommendations for the evaluation of technology-enhanced teacher professional development workshops are proposed.<\/jats:p>","DOI":"10.3390\/fi13030056","type":"journal-article","created":{"date-parts":[[2021,2,24]],"date-time":"2021-02-24T20:53:21Z","timestamp":1614200001000},"page":"56","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":16,"title":["Online Professional Learning in Response to COVID-19\u2014Towards Robust Evaluation"],"prefix":"10.3390","volume":"13","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-2018-3065","authenticated-orcid":false,"given":"Alireza","family":"Ahadi","sequence":"first","affiliation":[{"name":"School of Education, Faculty of Arts, Macquarie University, Macquarie Park, NSW 2109, Australia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4161-5816","authenticated-orcid":false,"given":"Matt","family":"Bower","sequence":"additional","affiliation":[{"name":"School of Education, Faculty of Arts, Macquarie University, Macquarie Park, NSW 2109, Australia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3783-6325","authenticated-orcid":false,"given":"Abhay","family":"Singh","sequence":"additional","affiliation":[{"name":"Department of Applied Finance, Macquarie Business School, Macquarie University, Macquarie Park, NSW 2109, Australia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0352-5008","authenticated-orcid":false,"given":"Michael","family":"Garrett","sequence":"additional","affiliation":[{"name":"Cinglevue International Pty Ltd., Unit 22\/1 Walsh Loop, Joondalup, WA 6027, Australia"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2021,2,24]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"279","DOI":"10.1002\/ase.1963","article-title":"Going virtual to support anatomy education: A STOPGAP in the midst of the Covid-19 pandemic","volume":"13","author":"Evans","year":"2020","journal-title":"Anat. Sci. Educ."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.jbusres.2020.05.034","article-title":"The future of business education: A commentary in the shadow of the Covid-19 pandemic","volume":"117","author":"Krishnamurthy","year":"2020","journal-title":"J. Bus. Res."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"729","DOI":"10.1016\/j.jsurg.2020.03.018","article-title":"Using technology to maintain the education of residents during the COVID-19 pandemic","volume":"77","author":"Chick","year":"2020","journal-title":"J. Surg. Educ."},{"key":"ref_4","unstructured":"Burgess, S., and Sievertsen, H.H. (2021, February 19). Schools, Skills, and Learning: The Impact of COVID-19 on Education. Available online: https:\/\/voxeu.org\/article\/impact-covid-19-education."},{"key":"ref_5","unstructured":"Marinoni, G., Van\u2019t Land, H., and Jensen, T. (2020). The Impact of COVID-19 on Higher Education around the World, IAU. IAU Global Survey Report."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"923","DOI":"10.1007\/s42438-020-00155-y","article-title":"Online university teaching during and after the Covid-19 crisis: Refocusing teacher presence and learning activity","volume":"2","author":"Rapanta","year":"2020","journal-title":"Postdigit. Sci. Educ."},{"key":"ref_7","first-page":"1","article-title":"The difference between emergency remote teaching and online learning","volume":"27","author":"Hodges","year":"2020","journal-title":"Educ. Rev."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"507","DOI":"10.1080\/02607476.2020.1799709","article-title":"Teacher education in times of COVID-19 pandemic in Portugal: National, institutional and pedagogical responses","volume":"46","author":"Gago","year":"2020","journal-title":"J. Educ. Teach."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"609","DOI":"10.1080\/02607476.2020.1755205","article-title":"Adaptations to a face-to-face initial teacher education course \u2018forced\u2019online due to the COVID-19 pandemic","volume":"46","author":"Moorhouse","year":"2020","journal-title":"J. Educ. Teach."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"596","DOI":"10.1080\/02607476.2020.1803051","article-title":"Initial teacher education in England and the Covid-19 pandemic: Challenges and opportunities","volume":"46","author":"Newman","year":"2020","journal-title":"J. Educ. Teach."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"5","DOI":"10.3102\/0013189X015005005","article-title":"Staff development and the process of teacher change","volume":"15","author":"Guskey","year":"1986","journal-title":"Educ. Res."},{"key":"ref_12","unstructured":"Kirkpatrick, D., and Kirkpatrick, J. (2006). Evaluating Training Programs: The Four Levels, Berrett-Koehler Publishers."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"947","DOI":"10.1016\/S0742-051X(02)00053-7","article-title":"Elaborating a model of teacher professional growth","volume":"18","author":"Clarke","year":"2002","journal-title":"Teach. Teach. Educ."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Ahadi, A., Bower, M., Singh, A., and Garrett, M. (2021). Evaluation of Teacher Professional Learning Workshops on the Use of Technology\u2014A Systematic Review. Prof. Dev. Educ., under review.","DOI":"10.1080\/19415257.2021.2011773"},{"key":"ref_15","first-page":"111","article-title":"The importance of educational technology in teaching","volume":"3","author":"Lazar","year":"2015","journal-title":"Int. J. Cogn. Res. Sci. Eng. Educ."},{"key":"ref_16","first-page":"210","article-title":"Expanding the Technology Acceptance Model (TAM) to examine faculty use of Learning Management Systems (LMSs) in higher education institutions","volume":"11","author":"Fathema","year":"2015","journal-title":"J. Online Learn. Teach."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"661","DOI":"10.1016\/S0883-0355(03)00064-8","article-title":"The architecture of professional development: Materials, messages and meaning","volume":"37","author":"Bredeson","year":"2002","journal-title":"Int. J. Educ. Res."},{"key":"ref_18","unstructured":"Sparks, D. (2002). Designing Powerful Professional Development for Teachers and Principals, National Staff Development Council."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"255","DOI":"10.1080\/15391523.2010.10782551","article-title":"Teacher technology change: How knowledge, confidence, beliefs, and culture intersect","volume":"42","author":"Ertmer","year":"2010","journal-title":"J. Res. Technol. Educ."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Duke, D.L., and Stiggins, R.J. (1990). Beyond minimum competence: Evaluation for professional development. The New Handbook of Teacher Evaluation: Assessing Elementary and Secondary School Teachers, Corwin Press.","DOI":"10.4135\/9781412986250.n7"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"341","DOI":"10.1016\/j.evalprogplan.2004.04.011","article-title":"A critical analysis of evaluation practice: The Kirkpatrick model and the principle of beneficence","volume":"27","author":"Bates","year":"2004","journal-title":"Eval. Progr. Plan."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"573","DOI":"10.1177\/1098214007308902","article-title":"Evaluation Theory, Models, and Applications, by Daniel L. Stufflebeam and Anthony J. Shinkfield. San Francisco: Jossey-Bass, 2007. 768pp","volume":"28","author":"Datta","year":"2007","journal-title":"Am. J. Eval."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"37","DOI":"10.1177\/1523422304272080","article-title":"Holton\u2019s evaluation model: New evidence and construct elaborations","volume":"7","author":"Holton","year":"2005","journal-title":"Adv. Dev. Hum. Resour."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"181","DOI":"10.3102\/0013189X08331140","article-title":"Improving impact studies of teachers\u2019 professional development: Toward better conceptualizations and measures","volume":"38","author":"Desimone","year":"2009","journal-title":"Educ. Res."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"27","DOI":"10.1016\/j.compedu.2019.01.010","article-title":"How is the use of technology in education evaluated? A systematic review","volume":"133","author":"Lai","year":"2019","journal-title":"Comput. Educ."},{"key":"ref_26","unstructured":"Laurillard, D. (2013). Teaching as a Design Science: Building Pedagogical Patterns for Learning and Technology, Routledge."},{"key":"ref_27","unstructured":"Yin, R.K. (2017). Case Study Research and Applications: Design and Methods, Sage Publications."},{"key":"ref_28","first-page":"134","article-title":"Three approaches to case study methods in education: Yin, Merriam, and Stake","volume":"20","author":"Yazan","year":"2015","journal-title":"Qual. Rep."},{"key":"ref_29","unstructured":"Stufflebeam, D.L., and Coryn, C.L. (2014). Evaluation Theory, Models, and Applications, John Wiley & Sons."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Stufflebeam, D.L., and Shinkfield, A.J. (1985). An analysis of alternative approaches to evaluation. Systematic Evaluation, Springer.","DOI":"10.1007\/978-94-009-5656-8"}],"container-title":["Future Internet"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1999-5903\/13\/3\/56\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T05:27:45Z","timestamp":1760160465000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1999-5903\/13\/3\/56"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,2,24]]},"references-count":30,"journal-issue":{"issue":"3","published-online":{"date-parts":[[2021,3]]}},"alternative-id":["fi13030056"],"URL":"https:\/\/doi.org\/10.3390\/fi13030056","relation":{},"ISSN":["1999-5903"],"issn-type":[{"value":"1999-5903","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,2,24]]}}}