{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,9]],"date-time":"2026-07-09T00:05:17Z","timestamp":1783555517410,"version":"3.55.0"},"reference-count":108,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2023,7,3]],"date-time":"2023-07-03T00:00:00Z","timestamp":1688342400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100002347","name":"Bundesministerium f\u00fcr Bildung und Forschung","doi-asserted-by":"publisher","id":[{"id":"10.13039\/501100002347","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Artif. Intell."],"abstract":"<jats:sec><jats:title>Introduction<\/jats:title><jats:p>Adaptive learning opportunities and individualized, timely feedback are considered to be effective support measures for students' writing in educational contexts. However, the extensive time and expertise required to analyze numerous drafts of student writing pose a barrier to teaching. Automated writing evaluation (AWE) tools can be used for individual feedback based on advances in Artificial Intelligence (AI) technology. A number of primary (quasi-)experimental studies have investigated the effect of AWE feedback on students' writing performance.<\/jats:p><\/jats:sec><jats:sec><jats:title>Methods<\/jats:title><jats:p>This paper provides a meta-analysis of the effectiveness of AWE feedback tools. The literature search yielded 4,462 entries, of which 20 studies (<jats:italic>k<\/jats:italic> = 84; <jats:italic>N<\/jats:italic> = 2, 828) met the pre-specified inclusion criteria. A moderator analysis investigated the impact of the characteristics of the learner, the intervention, and the outcome measures.<\/jats:p><\/jats:sec><jats:sec><jats:title>Results<\/jats:title><jats:p>Overall, results based on a three-level model with random effects show a medium effect (<jats:italic>g<\/jats:italic> = 0.55) of automated feedback on students' writing performance. However, the significant heterogeneity in the data indicates that the use of automated feedback tools cannot be understood as a single consistent form of intervention. Even though for some of the moderators we found substantial differences in effect sizes, none of the subgroup comparisons were statistically significant.<\/jats:p><\/jats:sec><jats:sec><jats:title>Discussion<\/jats:title><jats:p>We discuss these findings in light of automated feedback use in educational practice and give recommendations for future research.<\/jats:p><\/jats:sec>","DOI":"10.3389\/frai.2023.1162454","type":"journal-article","created":{"date-parts":[[2023,7,3]],"date-time":"2023-07-03T07:33:16Z","timestamp":1688369596000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":98,"title":["Automated feedback and writing: a multi-level meta-analysis of effects on students' performance"],"prefix":"10.3389","volume":"6","author":[{"given":"Johanna","family":"Fleckenstein","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Lucas W.","family":"Liebenow","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jennifer","family":"Meyer","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1965","published-online":{"date-parts":[[2023,7,3]]},"reference":[{"key":"B1","first-page":"316","article-title":"\u201cComputer-based writing instruction,\u201d","author":"Allen","year":"2016","journal-title":"Handbook of Writing Research"},{"key":"B2","doi-asserted-by":"publisher","first-page":"33","DOI":"10.1108\/ITSE-03-2020-0033","article-title":"The effectiveness of integrating technology in EFL\/ESL writing: A meta-analysis","volume":"2020","author":"Al-Wasy","year":"2020","journal-title":"Interact. Technol. Smart Educ"},{"key":"B3","doi-asserted-by":"publisher","first-page":"154","DOI":"10.20982\/tqmp.12.3.p154","article-title":"Fitting three-level meta-analytic models in R: A step-by-step tutorial","volume":"12","author":"Assink","year":"2016","journal-title":"Quantit. Methods Psychol"},{"key":"B4","doi-asserted-by":"crossref","first-page":"142","DOI":"10.4324\/9781315871493-8","article-title":"\u201cValidity and automated scoring,\u201d","volume-title":"Technology and Testing","author":"Bennett","year":"2015"},{"key":"B5","doi-asserted-by":"publisher","first-page":"5","DOI":"10.1007\/s11092-008-9068-5","article-title":"Developing the theory of formative assessment","volume":"21","author":"Black","year":"2009","journal-title":"Educat. Assess. Eval. Accountabil"},{"key":"B6","doi-asserted-by":"publisher","first-page":"427","DOI":"10.21240\/mpaed\/jb17\/2020.05.17.X","article-title":"Lernunterst\u00fctzung mit digitalen Unterrichtsmaterialien: Interdisziplin\u00e4re Erkenntnisse und Entwicklungsperspektiven","volume":"17","author":"B\u00f6hme","year":"2020","journal-title":"MedienP\u00e4dagogik"},{"key":"B7","doi-asserted-by":"publisher","first-page":"929","DOI":"10.1007\/s11145-015-9604-x","article-title":"Third and fourth grade teacher's classroom practices in writing: A national survey","volume":"29","author":"Brindle","year":"2015","journal-title":"Read. Writ"},{"key":"B8","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0235209","article-title":"Assisting students' writing with computer-based concept map feedback: A validation study of the CohViz feedback system","author":"Burkhart","year":"2020","journal-title":"PLoS ONE"},{"key":"B9","doi-asserted-by":"publisher","first-page":"18","DOI":"10.1016\/j.iheduc.2017.04.002","article-title":"The impact of online automated feedback on students' reflective journal writing in an EFL course","volume":"34","author":"Cheng","year":"2017","journal-title":"Internet High. Educ"},{"key":"B10","doi-asserted-by":"publisher","first-page":"211","DOI":"10.1037\/a0032968","article-title":"Modeling dependent effect sizes with three-level meta-analyses: A structural equation modeling approach","volume":"19","author":"Cheung","year":"2014","journal-title":"Psychol. Methods"},{"key":"B11","doi-asserted-by":"publisher","first-page":"435","DOI":"10.1111\/jcal.12349","article-title":"The effects of a1 theory-based summary writing tool on students' summary writing","volume":"35","author":"Chew","year":"2019","journal-title":"J. Comput. Assist. Learn"},{"key":"B12","doi-asserted-by":"publisher","first-page":"101","DOI":"10.2307\/3001666","article-title":"The combination of estimates from different experiments","volume":"10","author":"Cochran","year":"1954","journal-title":"Biometrics"},{"key":"B13","doi-asserted-by":"publisher","first-page":"793","DOI":"10.1007\/s11145-015-9596-6","article-title":"Writing instruction in first grade: An observational study","volume":"29","author":"Coker","year":"2016","journal-title":"Read. Writ"},{"key":"B14","doi-asserted-by":"publisher","DOI":"10.1016\/j.caeo.2022.100084","article-title":"Building a validity argument for an automated writing evaluation system (eRevise) as a formative assessment","author":"Correnti","year":"2022","journal-title":"Comput Educ. Open."},{"key":"B15","doi-asserted-by":"publisher","DOI":"10.1016\/j.compedu.2020.104094","article-title":"A review of automated feedback systems for learners: Classification framework, challenges and opportunities","author":"Deeva","year":"2021","journal-title":"Comput Educ."},{"key":"B16","doi-asserted-by":"publisher","first-page":"153","DOI":"10.1080\/00273171.2020.1853501","article-title":"AI-assisted systematic reviewing: Selecting studies to compare bayesian versus frequentist SEM for small sample sizes","volume":"56","author":"Ferdinands","year":"2021","journal-title":"Multivar. Behav. Res"},{"key":"B17","doi-asserted-by":"publisher","first-page":"100420","DOI":"10.1016\/j.asw.2019.100420","article-title":"Linking TOEFL iBT\u00ae writing rubrics to CEFR levels: Cut scores and validity evidence from a standard setting study","volume":"43","author":"Fleckenstein","year":"","journal-title":"Assess. Writ"},{"key":"B18","doi-asserted-by":"publisher","first-page":"90","DOI":"10.1080\/15434303.2017.1421956","article-title":"Teachers' judgement accuracy concerning CEFR levels of prospective university students","volume":"15","author":"Fleckenstein","year":"2018","journal-title":"Lang. Assess. Quart"},{"key":"B19","doi-asserted-by":"publisher","first-page":"562462","DOI":"10.3389\/fpsyg.2020.562462","article-title":"Is a long essay always a good essay? The effect of text length on writing assessment","volume":"11","author":"Fleckenstein","year":"","journal-title":"Front. Psychol"},{"key":"B20","doi-asserted-by":"publisher","first-page":"3","DOI":"10.1007\/978-3-658-37895-0_1","article-title":"\u201cDigitale Schreibf\u00f6rderung im Bildungskontext: Ein systematisches Review,\u201d","author":"Fleckenstein","year":"2023","journal-title":"Bildung f\u00fcr eine digitale Zukunft, Vol. 15"},{"key":"B21","doi-asserted-by":"publisher","first-page":"27","DOI":"10.14786\/flr.v3i2.162","article-title":"What works in school? Expert and novice teachers' beliefs about school effectiveness","volume":"3","author":"Fleckenstein","year":"2015","journal-title":"Frontl. Learn. Res"},{"key":"B22","doi-asserted-by":"publisher","first-page":"23","DOI":"10.3102\/978-0-935302-48-6_23","article-title":"Teaching writing in a digital and global age: Toward access, learning, and development for all","volume":"6","author":"Freedman","year":"2016","journal-title":"Am. Educ. Res. Assoc"},{"key":"B23","first-page":"65","article-title":"The effect of two forms of computer-automated metalinguistic corrective feedback","volume":"23","author":"Gao","year":"2019","journal-title":"Lang. Learn. Technol."},{"key":"B24","doi-asserted-by":"publisher","first-page":"277","DOI":"10.1177\/1077559504264265","article-title":"The effects of early prevention programs for families with young children at risk for physical child abuse and neglect: A meta-analysis","volume":"9","author":"Geeraert","year":"2004","journal-title":"Child Maltreat"},{"key":"B25","doi-asserted-by":"publisher","first-page":"258","DOI":"10.1080\/00461520.2018.1481406","article-title":"A revised writer (s)-within-community model of writing","volume":"53","author":"Graham","year":"2018","journal-title":"Educ. Psycholog"},{"key":"B26","doi-asserted-by":"publisher","first-page":"277","DOI":"10.3102\/0091732X18821125","article-title":"Changing how writing is taught","volume":"43","author":"Graham","year":"2019","journal-title":"Rev. Res. Educ"},{"key":"B27","doi-asserted-by":"publisher","first-page":"710","DOI":"10.17763\/haer.81.4.t2k0m13756113566","article-title":"Writing to read: A meta-analysis of the impact of writing and writing instruction on reading","volume":"81","author":"Graham","year":"2011","journal-title":"Harv. Educ. Rev"},{"key":"B28","doi-asserted-by":"publisher","first-page":"523","DOI":"10.1086\/681947","article-title":"Formative assessment and writing","volume":"115","author":"Graham","year":"2015","journal-title":"Element. School J"},{"key":"B29","doi-asserted-by":"publisher","first-page":"179","DOI":"10.3102\/0034654320914744","article-title":"The effects of writing on learning in science, social studies, and mathematics: A meta-analysis","volume":"90","author":"Graham","year":"2020","journal-title":"Rev. Educ. Res"},{"key":"B30","doi-asserted-by":"publisher","first-page":"781","DOI":"10.1007\/s11145-016-9640-1","article-title":"Writing education around the globe: Introduction and call for a new global analysis","volume":"29","author":"Graham","year":"2016","journal-title":"Read. Writ"},{"key":"B31","volume-title":"Effect Sizes for Research: A Broad Practical Approach","author":"Grissom","year":"2005"},{"key":"B32","doi-asserted-by":"publisher","first-page":"1494","DOI":"10.1111\/jcal.12587","article-title":"Implementing an automated feedback program for a foreign language writing course: A learner-centric study: Implementing an AWE tool in a L2 class","volume":"37","author":"Hassanzadeh","year":"2021","journal-title":"J. Comput. Assist. Learn"},{"key":"B33","first-page":"275","article-title":"The black box of tertiary assessment: An impending revolution","volume":"259","author":"Hattie","year":"2009","journal-title":"Tertiary Assess. High. Educ. Stud. Outcomes"},{"key":"B34","unstructured":"HattieJ.\n          Visible Learning Meta2022"},{"key":"B35","doi-asserted-by":"publisher","first-page":"81","DOI":"10.3102\/003465430298487","article-title":"The power of feedback","volume":"77","author":"Hattie","year":"2007","journal-title":"Rev. Educ. Res"},{"key":"B36","doi-asserted-by":"publisher","first-page":"107","DOI":"10.3102\/10769986006002107","article-title":"Distribution theory for glass's estimator of effect size and related estimators","volume":"6","author":"Hedges","year":"1981","journal-title":"J. Educ. Stat"},{"key":"B37","doi-asserted-by":"publisher","first-page":"39","DOI":"10.1002\/jrsm.5","article-title":"Robust variance estimation in meta-regression with dependent effect size estimates","volume":"1","author":"Hedges","year":"2010","journal-title":"Res. Synth. Methods"},{"key":"B38","doi-asserted-by":"publisher","DOI":"10.1558\/cj.v33i1.29251","article-title":"Automated writing evaluation in language teaching: Theory, development, and application","author":"Hegelheimer","year":"2016","journal-title":"CALICO J"},{"key":"B39","doi-asserted-by":"publisher","first-page":"1539","DOI":"10.1002\/sim.1186","article-title":"Quantifying heterogeneity in a meta-analysis","volume":"21","author":"Higgins","year":"2002","journal-title":"Stat. Med"},{"key":"B40","doi-asserted-by":"publisher","first-page":"82","DOI":"10.1093\/elt\/ccy044","article-title":"Automated writing evaluation","volume":"73","author":"Hockly","year":"2018","journal-title":"ELT J"},{"key":"B41","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1037\/bul0000354","article-title":"Which student and instructional variables are most strongly related to academic motivation in K-12 education? A systematic review of meta-analyses","volume":"148","author":"Jansen","year":"2022","journal-title":"Psychol. Bullet."},{"key":"B42","doi-asserted-by":"publisher","first-page":"103216","DOI":"10.1016\/j.tate.2020.103216","article-title":"Judgment accuracy in experienced vs. student teachers: Assessing essays in english as a foreign language","volume":"97","author":"Jansen","year":"2021","journal-title":"Teach. Teacher Educ"},{"key":"B43","doi-asserted-by":"publisher","DOI":"10.2378\/peu2019.art21d","article-title":"Das Sch\u00fclerinventar ASSET zur Beurteilung von Sch\u00fclerarbeiten im Fach Englisch: Drei experimentelle Studien zu Effekten der Textqualit\u00e4t und der Sch\u00fclernamen","author":"Jansen","year":"2019","journal-title":"Psychologie in Erziehung Und Unterricht"},{"key":"B44","doi-asserted-by":"publisher","first-page":"100700","DOI":"10.1016\/j.jslw.2019.100700","article-title":"English writing skills of students in upper secondary education: Results from an empirical study in Switzerland and Germany","volume":"48","author":"Keller","year":"2020","journal-title":"J. Second Lang. Writ"},{"key":"B45","doi-asserted-by":"publisher","first-page":"173","DOI":"10.2190\/EC.42.2.c","article-title":"Does automated feedback help students learn to write?","volume":"42","author":"Kellogg","year":"2010","journal-title":"J. Educ. Comput. Res"},{"key":"B46","doi-asserted-by":"publisher","first-page":"311","DOI":"10.17239\/jowr-2016.07.03.01","article-title":"Trends in research on writing as a learning activity","volume":"7","author":"Klein","year":"2016","journal-title":"J. Writ. Res"},{"key":"B47","doi-asserted-by":"publisher","first-page":"61","DOI":"10.1002\/jrsm.35","article-title":"Fixed effects and variance components estimation in three-level meta-analysis","volume":"2","author":"Konstantopoulos","year":"2011","journal-title":"Res. Synth. Methods"},{"key":"B48","doi-asserted-by":"publisher","first-page":"241","DOI":"10.3102\/0013189X20912798","article-title":"Interpreting effect sizes of education interventions","volume":"49","author":"Kraft","year":"2020","journal-title":"Educ. Research"},{"key":"B49","doi-asserted-by":"publisher","first-page":"29","DOI":"10.1037\/xap0000111","article-title":"Mind the gap! Automated concept map feedback supports students in writing cohesive explanations","volume":"23","author":"Lachner","year":"2017","journal-title":"J. Exp. Psychol"},{"key":"B50","doi-asserted-by":"publisher","first-page":"62","DOI":"10.1177\/0265532220929918","article-title":"Automated scoring of junior and senior high essays using Coh-Metrix features: Implications for large-scale language testing","volume":"38","author":"Latifi","year":"2021","journal-title":"Lang. Test"},{"key":"B51","doi-asserted-by":"crossref","DOI":"10.4159\/9780674417502","volume-title":"Making the Most of College","author":"Light","year":"2001"},{"key":"B52","first-page":"78","article-title":"Enhancing post-secondary writers' writing skills with a chatbot","volume":"23","author":"Lin","year":"2020","journal-title":"J. Educ. Technol. Soc"},{"key":"B53","doi-asserted-by":"publisher","first-page":"605","DOI":"10.1080\/09588221.2020.1743323","article-title":"Impact of automated writing evaluation on teacher feedback, student revision, and writing improvement","volume":"35","author":"Link","year":"2022","journal-title":"Comput. Assist. Lang. Learn"},{"key":"B54","volume-title":"Practical Meta-analysis","author":"Lipsey","year":"2001"},{"key":"B55","doi-asserted-by":"publisher","first-page":"121","DOI":"10.1089\/big.2018.0151","article-title":"An empirical study on the artificial intelligence writing evaluation system in china CET","volume":"7","author":"Lu","year":"2019","journal-title":"Big Data"},{"key":"B56","doi-asserted-by":"publisher","DOI":"10.5281\/zenodo.1249218","author":"L\u00fcdecke","year":"2019","journal-title":"ESC: Effect Size Computation for Meta-analysis (Version 0.5.1)"},{"key":"B57","doi-asserted-by":"publisher","first-page":"270","DOI":"10.1007\/978-3-030-23204-7_23","article-title":"\u201cChecking it twice: Does adding spelling and grammar checkers improve essay quality in an automated writing tutor?\u201d","author":"McCarthy","year":"2019","journal-title":"Artificial Intelligence in Education: 20th International Conference, AIED 2019, Chicago, IL, USA, June 25-29. 2019, Proceedings, Part i 20"},{"key":"B58","doi-asserted-by":"publisher","first-page":"35","DOI":"10.1016\/j.asw.2014.09.002","article-title":"A hierarchical classification approach to automated essay scoring","volume":"23","author":"McNamara","year":"2015","journal-title":"Assess. Writ"},{"key":"B59","doi-asserted-by":"publisher","DOI":"10.1037\/edu0000764","article-title":"Effects of computer-based feedback on lower- and higher-order learning outcomes: A network meta-analysis","author":"Mertens","year":"2022","journal-title":"J. Educ. Psychol."},{"key":"B60","doi-asserted-by":"publisher","first-page":"264","DOI":"10.7326\/0003-4819-151-4-200908180-00135","article-title":"Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement","volume":"151","author":"Moher","year":"2009","journal-title":"Ann. Internal Med"},{"key":"B61","first-page":"213","article-title":"EssayCritic: Writing to learn with a knowledge-based design critiquing system","volume":"20","author":"M\u00f8rch","year":"2017","journal-title":"J. Educ. Technol. Soc"},{"key":"B62","doi-asserted-by":"publisher","first-page":"364","DOI":"10.1177\/1094428106291059","article-title":"Estimating effect sizes from pretest-posttest-control group designs","volume":"11","author":"Morris","year":"2008","journal-title":"Organ. Res. Methods"},{"key":"B63","first-page":"745","article-title":"\u201cFeedback Research Revisited,\u201d","author":"Mory","year":"2004","journal-title":"Handbook of Research on Educational, Communications and Technology"},{"key":"B64","unstructured":"The Nation's Report Card: Writing 20112012"},{"key":"B65","unstructured":"Technical Summary of Preliminary Analyses of NAEP 2017 Writing Assessments2017"},{"key":"B66","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1080\/10494820.2022.2096642","article-title":"The effectiveness of automated writing evaluation in EFL\/ESL writing: A three-level meta-analysis","volume":"2022","author":"Ngo","year":"2022","journal-title":"Interact. Learn. Environ."},{"key":"B67","doi-asserted-by":"publisher","first-page":"599","DOI":"10.1111\/jcal.12635","article-title":"Effectiveness of automated writing evaluation systems in school settings: A systematic review of studies from 2000 to 2020","volume":"38","author":"Nunes","year":"2021","journal-title":"J. Comput. Assist. Learn"},{"key":"B68","doi-asserted-by":"publisher","first-page":"255","DOI":"10.1016\/j.cedpsych.2018.07.002","article-title":"Teacher implementation of self-regulated strategy development with an automated writing evaluation system: Effects on the argumentative writing performance of middle school students","volume":"54","author":"Palermo","year":"2018","journal-title":"Contempor. Educ. Psychol"},{"key":"B69","doi-asserted-by":"publisher","first-page":"104","DOI":"10.1016\/j.asw.2014.05.001","article-title":"When \u201cthe state of the art\u201d is counting words","volume":"21","author":"Perelman","year":"2014","journal-title":"Assess. Writ"},{"key":"B70","doi-asserted-by":"publisher","first-page":"56","DOI":"10.1007\/s40593-016-0122-z","article-title":"Assessing text-based writing of low-skilled college students","volume":"28","author":"Perin","year":"2018","journal-title":"Int. J. Artif. Intellig. Educ"},{"key":"B71","unstructured":"PustejovskyJ.\n          clubSandwich: Cluster-Robust (Sandwich) Variance Estimators With Small-Sample Corrections2022"},{"key":"B72","unstructured":"R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing2021"},{"key":"B73","doi-asserted-by":"publisher","first-page":"585","DOI":"10.1007\/s40299-021-00597-3","article-title":"Investigating the effects of perceived feedback source on second language writing performance: A quasi-experimental study","volume":"30","author":"Reynolds","year":"2021","journal-title":"Asia-Pacific Educ. Research"},{"key":"B74","doi-asserted-by":"publisher","first-page":"267","DOI":"10.2190\/U552-M54Q-5771-M677","article-title":"Experimental evidence on the effectiveness of automated essay scoring in teacher education cases","volume":"35","author":"Riedel","year":"2006","journal-title":"J. Educ. Comput. Res"},{"key":"B75","doi-asserted-by":"publisher","first-page":"d549","DOI":"10.1136\/bmj.d549","article-title":"Interpretation of random effects meta-analyses","volume":"342","author":"Riley","year":"2011","journal-title":"Br. Med. J."},{"key":"B76","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1002\/ets2.12249","article-title":"Automated essay scoring at scale: A case study in Switzerland and Germany","volume":"2019","author":"Rupp","year":"2019","journal-title":"ETS Res. Rep. Ser"},{"key":"B77","doi-asserted-by":"publisher","first-page":"204","DOI":"10.1016\/j.obhdp.2013.03.002","article-title":"Methods for second order meta-analysis and illustrative applications","volume":"121","author":"Schmidt","year":"2013","journal-title":"Org. Behav. Hum. Decision Process"},{"key":"B78","doi-asserted-by":"publisher","first-page":"53","DOI":"10.1016\/j.asw.2013.04.001","article-title":"State-of-the-art automated essay scoring: Competition, results, and future directions from a united states demonstration","volume":"20","author":"Shermis","year":"2014","journal-title":"Assess. Writ"},{"key":"B79","doi-asserted-by":"publisher","first-page":"153","DOI":"10.3102\/0034654307313795","article-title":"Focus on formative feedback","volume":"78","author":"Shute","year":"2008","journal-title":"Rev. Educ. Res"},{"key":"B80","doi-asserted-by":"publisher","first-page":"3681","DOI":"10.1016\/j.csda.2005.07.019","article-title":"Robust variance estimation for random effects meta-analysis","volume":"50","author":"Sidik","year":"2006","journal-title":"Comput. Stat. Data Anal"},{"key":"B81","doi-asserted-by":"publisher","first-page":"121","DOI":"10.1080\/0969594X.2022.2089488","article-title":"Formative writing assessment for change\u2014introduction to the special issue","volume":"29","author":"Skar","year":"2022","journal-title":"Assess. Educ. Principl. Pol. Practice"},{"key":"B82","doi-asserted-by":"publisher","first-page":"693","DOI":"10.1007\/s11618-018-0839-6","article-title":"Formatives assessment","volume":"21","author":"Souvignier","year":"2018","journal-title":"Zeitschrift f\u00fcr Erziehungswissenschaft."},{"key":"B83","doi-asserted-by":"publisher","first-page":"99","DOI":"10.1002\/0470870168.ch6","article-title":"Regression methods to detect publication and other bias in meta-analysis","volume":"6","author":"Sterne","year":"2005","journal-title":"Publicat. Bias Meta-Analysis"},{"key":"B84","doi-asserted-by":"publisher","first-page":"51","DOI":"10.1016\/j.asw.2013.11.007","article-title":"The effects of computer-generated feedback on the quality of writing","volume":"19","author":"Stevenson","year":"2014","journal-title":"Assess. Writ"},{"key":"B85","doi-asserted-by":"publisher","first-page":"33","DOI":"10.1016\/j.compedu.2018.12.005","article-title":"Digital support for academic writing: A review of technologies and pedagogies","volume":"131","author":"Strobl","year":"2019","journal-title":"Comput. Educ"},{"key":"B86","doi-asserted-by":"publisher","first-page":"117","DOI":"10.29140\/jaltcall.v13n2.215","article-title":"Automated writing evaluation in an EFL setting: Lessons from china","volume":"13","author":"Tang","year":"2017","journal-title":"JALT CALL J"},{"key":"B87","doi-asserted-by":"publisher","first-page":"604","DOI":"10.3102\/1076998615606099","article-title":"Small-sample adjustments for tests of moderators and model fit using robust variance estimation in meta-regression","volume":"40","author":"Tipton","year":"2015","journal-title":"J. Educ. Behav. Stat"},{"key":"B88","unstructured":"International Standard Classification of Education: ISCED 20112012"},{"key":"B89","doi-asserted-by":"publisher","first-page":"125","DOI":"10.1038\/s42256-020-00287-7","article-title":"An open source machine learning framework for efficient and transparent systematic reviews","volume":"3","author":"Van de Schoot","year":"2021","journal-title":"Nat. Machine Intellig"},{"key":"B90","doi-asserted-by":"publisher","first-page":"576","DOI":"10.3758\/s13428-012-0261-6","article-title":"Three-level meta-analysis of dependent effect sizes","volume":"45","author":"Van den Noortgate","year":"2013","journal-title":"Behav. Res. Methods"},{"key":"B91","doi-asserted-by":"publisher","first-page":"1274","DOI":"10.3758\/s13428-014-0527-2","article-title":"Meta-analysis of multiple outcomes: A multilevel approach","volume":"47","author":"Van den Noortgate","year":"2015","journal-title":"Behav. Res. Methods"},{"key":"B92","doi-asserted-by":"publisher","first-page":"475","DOI":"10.3102\/0034654314564881","article-title":"Effects of feedback in a computer-based learning environment on students' learning outcomes: A meta-analysis","volume":"85","author":"Van der Kleij","year":"2015","journal-title":"Rev. Educ. Res"},{"key":"B93","doi-asserted-by":"publisher","first-page":"261","DOI":"10.3102\/10769986030003261","article-title":"Bias and efficiency of meta-analytic variance estimators in the random-effects model","volume":"30","author":"Viechtbauer","year":"2005","journal-title":"J. Educ. Behav. Stat"},{"key":"B94","doi-asserted-by":"publisher","first-page":"1","DOI":"10.18637\/jss.v036.i03","article-title":"Conducting meta-analyses in R with the metafor package","volume":"36","author":"Viechtbauer","year":"2010","journal-title":"J. Stat. Softw"},{"key":"B95","doi-asserted-by":"publisher","first-page":"333","DOI":"10.1207\/s1532690xci2203_3","article-title":"Summary street: Interactive computer support for writing","volume":"22","author":"Wade-Stein","year":"2004","journal-title":"Cogn. Instruct"},{"key":"B96","doi-asserted-by":"publisher","first-page":"234","DOI":"10.1080\/09588221.2012.655300","article-title":"Exploring the impact of using automated writing evaluation in english as a foreign language university students' writing","volume":"26","author":"Wang","year":"2013","journal-title":"Comput. Assist. Lang. Learn"},{"key":"B97","doi-asserted-by":"publisher","first-page":"227","DOI":"10.1016\/j.compedu.2018.09.024","article-title":"Technology and writing: Review of research","volume":"128","author":"Williams","year":"2019","journal-title":"Comput. Educ"},{"key":"B98","unstructured":"WilsonD. B.\n          Unpublished manuscriptGeorge Mason UniversityFormulas Used by the Practical Meta-analysis Effect Size Calculator. Practical Meta-Analysis2016"},{"key":"B99","doi-asserted-by":"publisher","first-page":"94","DOI":"10.1016\/j.compedu.2016.05.004","article-title":"Automated essay evaluation software in english language arts classrooms: Effects on teacher feedback, student motivation, and writing quality","volume":"100","author":"Wilson","year":"2016","journal-title":"Comput. Educ"},{"key":"B100","doi-asserted-by":"publisher","first-page":"87","DOI":"10.1177\/0735633119830764","article-title":"Automated writing evaluation and feedback: Multiple metrics of efficacy","volume":"58","author":"Wilson","year":"2020","journal-title":"J. Educ. Comput. Res"},{"key":"B101","doi-asserted-by":"publisher","first-page":"3087","DOI":"10.3389\/fpsyg.2019.03087","article-title":"The power of feedback revisited: A meta-analysis of educational feedback research","volume":"10","author":"Wisniewski","year":"2020","journal-title":"Front. Psychol."},{"key":"B102","doi-asserted-by":"publisher","first-page":"516","DOI":"10.1111\/jcal.12100","article-title":"The impact of computer-based instruction on the development of EFL learners' writing skills","volume":"31","author":"Zaini","year":"2015","journal-title":"J. Comput. Assist. Learn"},{"key":"B103","doi-asserted-by":"publisher","first-page":"171","DOI":"10.1186\/s41239-019-0171-0","article-title":"Systematic review of research on artificial intelligence applications in higher education\u2014where are the educators?","volume":"16","author":"Zawacki-Richter","year":"2019","journal-title":"Int. J. Educ. Technol. High. Educ."},{"key":"B104","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1080\/09588221.2021.1897019","article-title":"Automated writing evaluation (AWE) feedback: A systematic investigation of college students' acceptance","volume":"2021","author":"Zhai","year":"2021","journal-title":"Comput. Assist. Lang. Learn."},{"key":"B105","doi-asserted-by":"publisher","first-page":"7356331221127300","DOI":"10.1177\/07356331221127300","article-title":"The effectiveness of automated writing evaluation on writing quality: A meta-analysis","volume":"2022","author":"Zhai","year":"2022","journal-title":"J. Educ. Comput. Res"},{"key":"B106","doi-asserted-by":"publisher","first-page":"170","DOI":"10.1515\/jccall-2021-2007","article-title":"Review of automated writing evaluation systems","volume":"1","author":"Zhang","year":"2021","journal-title":"J. China Comput. Assist. Lang. Learn"},{"key":"B107","doi-asserted-by":"publisher","first-page":"103668","DOI":"10.1016\/j.compedu.2019.103668","article-title":"The effect of automated feedback on revision behavior and learning gains in formative assessment of scientific argument writing","volume":"143","author":"Zhu","year":"2020","journal-title":"Comput. Educ"},{"key":"B108","doi-asserted-by":"publisher","first-page":"21","DOI":"10.1007\/s10212-014-0225-6","article-title":"Teachers' judgements of students' foreign-language achievement","volume":"30","author":"Zhu","year":"2015","journal-title":"Eur. J. Psychol. Educ"}],"container-title":["Frontiers in Artificial Intelligence"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/frai.2023.1162454\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,7,3]],"date-time":"2023-07-03T07:33:29Z","timestamp":1688369609000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/frai.2023.1162454\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,7,3]]},"references-count":108,"alternative-id":["10.3389\/frai.2023.1162454"],"URL":"https:\/\/doi.org\/10.3389\/frai.2023.1162454","relation":{},"ISSN":["2624-8212"],"issn-type":[{"value":"2624-8212","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,7,3]]},"article-number":"1162454"}}