{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,4]],"date-time":"2026-03-04T19:29:13Z","timestamp":1772652553845,"version":"3.50.1"},"reference-count":96,"publisher":"Elsevier BV","issue":"1","license":[{"start":{"date-parts":[[2016,12,5]],"date-time":"2016-12-05T00:00:00Z","timestamp":1480896000000},"content-version":"tdm","delay-in-days":0,"URL":"http:\/\/www.springer.com\/tdm"},{"start":{"date-parts":[[2016,12,5]],"date-time":"2016-12-05T00:00:00Z","timestamp":1480896000000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/www.springer.com\/tdm"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Int J Artif Intell Educ"],"published-print":{"date-parts":[[2018,3]]},"DOI":"10.1007\/s40593-016-0128-6","type":"journal-article","created":{"date-parts":[[2016,12,5]],"date-time":"2016-12-05T15:02:13Z","timestamp":1480950133000},"page":"29-55","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":9,"title":["Wise Crowd Content Assessment and Educational Rubrics"],"prefix":"10.1016","volume":"28","author":[{"given":"Rebecca J.","family":"Passonneau","sequence":"first","affiliation":[]},{"given":"Ananya","family":"Poddar","sequence":"additional","affiliation":[]},{"given":"Gaurav","family":"Gite","sequence":"additional","affiliation":[]},{"given":"Alisa","family":"Krivokapic","sequence":"additional","affiliation":[]},{"given":"Qian","family":"Yang","sequence":"additional","affiliation":[]},{"given":"Dolores","family":"Perin","sequence":"additional","affiliation":[]}],"member":"78","published-online":{"date-parts":[[2016,12,5]]},"reference":[{"key":"128_CR1","unstructured":"Attali, Y., & Burstein, J. (2006). Automated essay scoring with e-rater\u00ae V. 2. Journal of Technology Learning and Assessment, 4(3), 3\u201339."},{"issue":"1","key":"128_CR2","doi-asserted-by":"publisher","first-page":"29","DOI":"10.3102\/00346543074001029","volume":"74","author":"RL Bangert-Drowns","year":"2004","unstructured":"Bangert-Drowns, R.L., Hurley, M.M., & Wilkinson, B. (2004). The effects of school-based writing-to-learn interventions on academic achievement: A meta-analysis. Review of Educational Research, 74(1), 29\u201358.","journal-title":"Review of Educational Research"},{"key":"128_CR3","doi-asserted-by":"publisher","first-page":"185","DOI":"10.1007\/s11145-007-9107-5","volume":"22","author":"SF Beers","year":"2009","unstructured":"Beers, S.F., & Nagy, W.E. (2009). Syntactic complexity as a predictor of adolescent writing quality: Which measures? which genre? Reading and Writing, 22, 185\u2013200.","journal-title":"Reading and Writing"},{"key":"128_CR4","doi-asserted-by":"publisher","first-page":"183","DOI":"10.1007\/s11145-010-9264-9","volume":"24","author":"SF Beers","year":"2011","unstructured":"Beers, S.F., & Nagy, W.E. (2011). Writing development in four genres from grades three to seven: syntactic complexity and genre differentiation. Reading and Writing, 24, 183\u2013202.","journal-title":"Reading and Writing"},{"key":"128_CR5","unstructured":"Beigman-Klebanov, B. (2015). Towards automated evaluation of writing along STEM-relevant dimensions. MARWiSE: Multidisciplinary Advances in Reading and Writing for Science Education Workshop. May 7-8, 2015, Columbia University."},{"key":"128_CR6","unstructured":"Beigman-Klebanov, B., Madnani, N., Burstein, J., & Somasundaran, S. (2014). Content importance models for scoring writing from sources. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 247\u2013252."},{"issue":"5","key":"128_CR7","doi-asserted-by":"publisher","first-page":"765","DOI":"10.1002\/sce.20402","volume":"94","author":"LK Berland","year":"2010","unstructured":"Berland, L.K., & McNeill, K.L. (2010). A learning progression for scientific argumentation: Understanding student work and designing supportive instructional contexts. Science Education, 94(5), 765\u2013793.","journal-title":"Science Education"},{"key":"128_CR8","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1016\/S0022-5371(83)80002-4","volume":"22","author":"AL Brown","year":"1983","unstructured":"Brown, A.L., & Day, J.D. (1983). Macrorules for summarizing texts: The development of expertise. Journal of Verbal Learning and Verbal Behavior, 22, 1\u201314.","journal-title":"Journal of Verbal Learning and Verbal Behavior"},{"key":"128_CR9","doi-asserted-by":"publisher","first-page":"968","DOI":"10.2307\/1129901","volume":"54","author":"AL Brown","year":"1983","unstructured":"Brown, A.L., Day, J.D., & Jones, R.S. (1983). The development of plans for summarizing texts. Child Development, 54, 968\u2013979.","journal-title":"Child Development"},{"key":"128_CR10","doi-asserted-by":"crossref","unstructured":"Burstein, J., Elliot, N., & Molloy, H. (2016). Informing automated writing evaluation using the lens of genre: Two studies. CALICO Journal, 33(1).","DOI":"10.1558\/cj.v33i1.26374"},{"key":"128_CR11","unstructured":"Burstein, J., Kukich, K., Wolff, S., Lu, C., & Chodorow, M. (1998). Enriching automated essay scoring using discourse marking. In Stede, M., Wanner, L., & Hovy, E. (Eds.) Workshop on Discourse Relations and Discourse Marking, pages 15\u201321. Association for Computational Linguistics."},{"issue":"3","key":"128_CR12","doi-asserted-by":"publisher","first-page":"277","DOI":"10.1207\/S1532690XCI1903_2","volume":"19","author":"KR Butcher","year":"2001","unstructured":"Butcher, K.R., & Kintsch, W. (2001). Support of content and rhetorical processes of writing: Effects on the writing process and the written product. Cognition and Instruction, 19(3), 277\u2013322.","journal-title":"Cognition and Instruction"},{"issue":"4","key":"128_CR13","doi-asserted-by":"publisher","first-page":"284","DOI":"10.1037\/1040-3590.6.4.284","volume":"6","author":"DV Cicchetti","year":"1994","unstructured":"Cicchetti, D.V. (1994). Guidelines, criteria, and rules of thumb for evaluating normed and standardized assessment instruments in psychology. Psychologial Assessment, 6(4), 284\u2013290.","journal-title":"Psychologial Assessment"},{"key":"128_CR14","doi-asserted-by":"crossref","unstructured":"Corro, L.D., & Gemulla, R. (2013). ClausIE: Clause-based open information extraction. In Proceedings of the 22nd International Conference on World Wide Web (WWW \u201913), pages 355\u2013366.","DOI":"10.1145\/2488388.2488420"},{"issue":"3","key":"128_CR15","doi-asserted-by":"publisher","first-page":"193","DOI":"10.1207\/s1532690xci0303_3","volume":"3","author":"JD Day","year":"1986","unstructured":"Day, J.D. (1986). Teaching summarization skills: Influences of student ability level and strategy difficulty. Cognition and Instruction, 3(3), 193\u2013210.","journal-title":"Cognition and Instruction"},{"issue":"1","key":"128_CR16","doi-asserted-by":"publisher","first-page":"7","DOI":"10.1016\/j.asw.2012.10.002","volume":"18","author":"P Deane","year":"2013","unstructured":"Deane, P. (2013). On the relation between automated essay scoring and modern views of the writing construct. Assessing Writing, 18(1), 7\u201324.","journal-title":"Assessing Writing"},{"key":"128_CR17","doi-asserted-by":"crossref","unstructured":"Deane, P., Odendahl, N., Quinlan, T., Fowles, M., Welsh, C., & Bivens-Tatum, J. (2008). Cognitive models of writing: Writing proficiency as a complex integrated skill. Technical Report 2, ETS Research Report Series, Princeton, NJ.","DOI":"10.1002\/j.2333-8504.2008.tb02141.x"},{"key":"128_CR18","doi-asserted-by":"crossref","unstructured":"Fellbaum, C. (Ed.) (1998). WordNet: An Electronic Lexical Database. MIT Press, Cambridge, MA.","DOI":"10.7551\/mitpress\/7287.001.0001"},{"key":"128_CR19","unstructured":"Foltz, P.W., Laham, D., & Landauer, T.K. (1999). The Intelligent Essay Assessor: Applications to educational technology. Interactive Multimedia Electronic Journal of Computer-Enhanced Learning, 1(2)."},{"issue":"4","key":"128_CR20","doi-asserted-by":"publisher","first-page":"549","DOI":"10.3102\/00028312022004549","volume":"22","author":"R Garner","year":"1985","unstructured":"Garner, R. (1985). Text summarization deficiencies among older students: Awareness or production ability American Educational Research Journal, 22(4), 549\u2013560.","journal-title":"American Educational Research Journal"},{"issue":"1","key":"128_CR21","doi-asserted-by":"publisher","first-page":"60","DOI":"10.1037\/edu0000052","volume":"108","author":"LF Gerard","year":"2016","unstructured":"Gerard, L.F., Ryoo, K., McElhaney, K.W., Liu, O.L., Rafferty, A.N., & Linn, M.C. (2016). Automated guidance for student inquiry. Journal of Educational Psychology, 108(1), 60\u201381.","journal-title":"Journal of Educational Psychology"},{"issue":"1","key":"128_CR22","doi-asserted-by":"publisher","first-page":"30","DOI":"10.1080\/02702710902733600","volume":"31","author":"L Gil","year":"2010","unstructured":"Gil, L., Bra\u0307ten, I., Vidal-Abarca, E., & Str\u00f8ms\u00f8, H.I. (2010). Understanding and integrating multiple science texts: Summary tasks are sometimes better than argument tasks. Reading Psychology, 31(1), 30\u201368.","journal-title":"Reading Psychology"},{"issue":"6","key":"128_CR23","doi-asserted-by":"publisher","first-page":"1043","DOI":"10.1007\/s11145-013-9494-8","volume":"27","author":"A Gillespie","year":"2014","unstructured":"Gillespie, A., Graham, S., Kiuhara, S., & Hebert, M. (2014). High school teachers\u2019 use of writing to support students\u2019 learning: a national survey. Reading and Writing, 27(6), 1043\u20131072.","journal-title":"Reading and Writing"},{"key":"128_CR24","volume-title":"The nation\u2019s report card: Reading 2009. Technical Report NCES 2010-458","author":"A Glymph","year":"2010","unstructured":"Glymph, A. (2010). The nation\u2019s report card: Reading 2009. Technical Report NCES 2010-458. Washington: National Center for Education Statistics (NCES)."},{"key":"128_CR25","volume-title":"The nation\u2019s report card: Reading 2012. Technical Report NCES 2012-457","author":"A Glymph","year":"2013","unstructured":"Glymph, A. (2013). The nation\u2019s report card: Reading 2012. Technical Report NCES 2012-457. Washington: National Center for Education Statistics (NCES)."},{"key":"128_CR26","volume-title":"The nation\u2019s report card: A first look: 2013 mathematics and reading. Technical Report NCES 2014-451","author":"A Glymph","year":"2013","unstructured":"Glymph, A., & Burg, S. (2013). The nation\u2019s report card: A first look: 2013 mathematics and reading. Technical Report NCES 2014-451. Washington: National Center for Education Statistics (NCES)."},{"issue":"6","key":"128_CR27","doi-asserted-by":"publisher","first-page":"1015","DOI":"10.1007\/s11145-013-9495-7","volume":"27","author":"S Graham","year":"2014","unstructured":"Graham, S., Capizzi, A., Harris, K., Hebert, M., & Morphy, P. (2014). Teaching writing to middle school students: a national survey. Reading and Writing, 27(6), 1015\u20131042.","journal-title":"Reading and Writing"},{"key":"128_CR28","doi-asserted-by":"crossref","unstructured":"Graham, S., & Perin, D. (2007a). A meta-analysis of writing instruction for adolescent students. Journal of Educational Psychology, 99(3), 445\u2013476.","DOI":"10.1037\/0022-0663.99.3.445"},{"key":"128_CR29","unstructured":"Graham, S., & Perin, D. (2007b). Writing next: Effective strategies to improve writing of adolescents in middle and high schools. New York: Technical report, Carnegie Corporation of New York."},{"key":"128_CR30","unstructured":"Guo, W., & Diab, M. (2012). Modeling sentences in the latent space. In Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics (ACL), pages 864\u2013872."},{"issue":"2","key":"128_CR31","doi-asserted-by":"publisher","first-page":"186","DOI":"10.1002\/tea.10128","volume":"41","author":"BM Hand","year":"2004","unstructured":"Hand, B.M., Hohenshell, L., & Prain, V. (2004). Exploring students\u2019 responses to conceptual questions when engaged with planned writing experiences: A study with year 10 science students. Journal of Research in Science Teaching, 41(2), 186\u2013210.","journal-title":"Journal of Research in Science Teaching"},{"key":"128_CR32","doi-asserted-by":"crossref","unstructured":"Hughes, S., Hastings, P., Magliano, J., Goldman, S., & Lawless, K. (2012). Automated approaches for detecting integration in student essays. In Automated approaches for detecting integration in student essays. Springer-Verlag.","DOI":"10.1007\/978-3-642-30950-2_35"},{"issue":"1","key":"128_CR33","doi-asserted-by":"publisher","first-page":"12","DOI":"10.1016\/S0022-5371(70)80003-2","volume":"9","author":"RE Johnson","year":"1970","unstructured":"Johnson, R.E. (1970). Recall of prose as a function of the structural importance of the linguistic units. Journal of Verbal Learning and Verbal Behavior, 9(1), 12\u201320.","journal-title":"Journal of Verbal Learning and Verbal Behavior"},{"issue":"1","key":"128_CR34","doi-asserted-by":"publisher","first-page":"1","DOI":"10.17239\/jowr-2008.01.01.1","volume":"1","author":"RT Kellogg","year":"2008","unstructured":"Kellogg, R.T. (2008). Training writing skills: a cognitive development perspective. Journal of Writing Research, 1(1), 1\u201326.","journal-title":"Journal of Writing Research"},{"key":"128_CR35","doi-asserted-by":"crossref","unstructured":"Kharkwal, G., & Muresan, S. (2014). Surprisal as a predictor of essay quality. In Proceedings of the Ninth Workshop on Innovative Use of NLP for Building Educational Applications, pages 54\u201360.","DOI":"10.3115\/v1\/W14-1807"},{"key":"128_CR36","doi-asserted-by":"crossref","unstructured":"Kintsch, W., & van Dijk, T.A. (1978). Toward a model of text comprehension and production. Psychological Review, 85, 364\u2013394.","DOI":"10.1037\/0033-295X.85.5.363"},{"issue":"4","key":"128_CR37","doi-asserted-by":"publisher","first-page":"433","DOI":"10.1598\/RRQ.45.4.4","volume":"45","author":"PD Klein","year":"2010","unstructured":"Klein, P.D., & Rose, M.A. (2010). Teaching argument and explanation to prepare junior students for writing to learn. Reading Research Quarterly, 45(4), 433\u2013461.","journal-title":"Reading Research Quarterly"},{"key":"128_CR38","volume-title":"Content Analysis: An Introduction to Its Methodology","author":"K Krippendorff","year":"1980","unstructured":"Krippendorff, K. (1980). Content Analysis: An Introduction to Its Methodology. Beverly Hills: Sage Publications."},{"key":"128_CR39","doi-asserted-by":"publisher","first-page":"83","DOI":"10.1002\/nav.3800020109","volume":"2","author":"HW Kuhn","year":"1955","unstructured":"Kuhn, H.W. (1955). The Hungarian Method for the assignment problem. Naval Research Logistics Quarterly, 2, 83\u201397.","journal-title":"Naval Research Logistics Quarterly"},{"issue":"4","key":"128_CR40","doi-asserted-by":"publisher","first-page":"389","DOI":"10.1023\/A:1025779619903","volume":"37","author":"C Leacock","year":"2003","unstructured":"Leacock, C., & Chodorow, M. (2003). C-rater: Automated scoring of short-answer questions. Computers and the Humanities, 37(4), 389\u2013405.","journal-title":"Computers and the Humanities"},{"key":"128_CR41","unstructured":"Lin, C.-Y. (2004). ROUGE: a package for automatic evaluation of summaries. In Proceedings of the Workshop on Text Summarization Branches Out (WAS 2004)."},{"issue":"2","key":"128_CR42","doi-asserted-by":"publisher","first-page":"19","DOI":"10.1111\/emip.12028","volume":"33","author":"OL Liu","year":"2014","unstructured":"Liu, O.L., Brew, C., Blackmore, J., Gerard, L.F., Madhok, J., & Linn, M.C. (2014). Automated scoring of constructed-response science items: Propsects and obstacles. Educational Measurement: Issues and Practice, 33(2), 19\u201328.","journal-title":"Educational Measurement: Issues and Practice"},{"key":"128_CR43","doi-asserted-by":"crossref","unstructured":"Louis, A., & Nenkova, A. (2009). Automatically evaluating content selection in summarization without human models. In Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing, pages 306\u2013314, Singapore. Association for Computational Linguistics.","DOI":"10.3115\/1699510.1699550"},{"issue":"2","key":"128_CR44","doi-asserted-by":"publisher","first-page":"267","DOI":"10.1162\/COLI_a_00123","volume":"39","author":"A Louis","year":"2013","unstructured":"Louis, A., & Nenkova, A. (2013). Automatically assessing machine summary content without a gold standard. Computational Linguistics, 39(2), 267\u2013300.","journal-title":"Computational Linguistics"},{"key":"128_CR45","doi-asserted-by":"crossref","unstructured":"Ma, X., & Hovy, E.H. (2016). End-to-end sequence labeling via bi-directional LSTM-CNNs-CRF. In Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL). To Appear.","DOI":"10.18653\/v1\/P16-1101"},{"key":"128_CR46","doi-asserted-by":"publisher","first-page":"615","DOI":"10.1037\/0022-0663.91.4.615","volume":"91","author":"JP Magliano","year":"1999","unstructured":"Magliano, J.P., Trabasso, T., & Graesser, A.C. (1999). Strategic processing during comprehension. Journal of Educational Psychology, 91, 615\u2013629.","journal-title":"Journal of Educational Psychology"},{"key":"128_CR47","volume-title":"Building bridges to college and careers: Contextualized basic skills programs at community colleges. Technical report","author":"C Mazzeo","year":"2003","unstructured":"Mazzeo, C., Rab, S.Y., & Alssid, J.L. (2003). Building bridges to college and careers: Contextualized basic skills programs at community colleges. Technical report. Brooklyn: Workforce Strategy Center."},{"key":"128_CR48","unstructured":"Mikolov, T., Sutskever, I., Chen, K., Corrado, G., & Dean, J. (2013). Distributed representations of words and phrases and their compositionality."},{"key":"128_CR49","unstructured":"NCES (2012). The nation\u2019s report card: Writing 2011."},{"key":"128_CR50","unstructured":"Nenkova, A., & Passonneau, R.J. (2004). Evaluating content selection in summarization: The pyramid method. In Susan Dumais, D.M., & Roukos, S. (Eds.) HLT-NAACL 2004: Main Proceedings, pages 145\u2013152, Boston, Massachusetts, USA. Association for Computational Linguistics."},{"key":"128_CR51","doi-asserted-by":"crossref","unstructured":"Nenkova, A., Passonneau, R.J., & McKeown, K. (2007). The pyramid method: incorporating human content selection variation in summarization evaluation. ACM Transactions on Speech and Language Processing, 4(2).","DOI":"10.1145\/1233912.1233913"},{"issue":"2","key":"128_CR52","doi-asserted-by":"publisher","first-page":"224","DOI":"10.1002\/sce.10066","volume":"87","author":"SP Norris","year":"2003","unstructured":"Norris, S.P., & Phillips, L.M. (2003). How literacy in its fundamental sense is central to scientific literacy. Science Education, 87(2), 224\u2013240.","journal-title":"Science Education"},{"issue":"2","key":"128_CR53","doi-asserted-by":"publisher","first-page":"391","DOI":"10.1037\/a0037549","volume":"107","author":"NG Olinghouse","year":"2015","unstructured":"Olinghouse, N.G., Graham, S., & Gillespie, A. (2015). The relationship of discourse and topic knowledge to fifth graders\u2019 writing performance. Journal of Educational Psychology, 107(2), 391\u2013406.","journal-title":"Journal of Educational Psychology"},{"issue":"1","key":"128_CR54","doi-asserted-by":"publisher","first-page":"45","DOI":"10.1007\/s11145-012-9392-5","volume":"26","author":"NG Olinghouse","year":"2013","unstructured":"Olinghouse, N.G., & Wilson, J. (2013). The relationship between vocabulary and writing quality in three genres. Reading and Writing, 26(1), 45\u201365.","journal-title":"Reading and Writing"},{"key":"128_CR55","unstructured":"Owczarzak, K., Conroy, J.M., Dang, H.T., & Nenkova, A. (2012). An assessment of the accuracy of automatic evaluation in summarization. In Proceedings of Workshop on Evaluation Metrics and System Comparison for Automatic Summarization, pages 1\u20139. Association for Computational Linguistics."},{"issue":"5","key":"128_CR56","first-page":"238","volume":"47","author":"EB Page","year":"1966","unstructured":"Page, E.B. (1966). The imminence of grading essays by computer. Phi Delta Kappan, 47(5), 238\u2013243.","journal-title":"Phi Delta Kappan"},{"key":"128_CR57","doi-asserted-by":"publisher","first-page":"210","DOI":"10.1007\/BF01419938","volume":"14","author":"EB Page","year":"1968","unstructured":"Page, E.B. (1968). The use of the computer in analyzing student essays. International Review of Education, 14, 210\u2013225.","journal-title":"International Review of Education"},{"issue":"2","key":"128_CR58","doi-asserted-by":"publisher","first-page":"127","DOI":"10.1080\/00220973.1994.9943835","volume":"62","author":"EB Page","year":"1994","unstructured":"Page, E.B. (1994). Computer grading of student prose, using modern concepts and software. The Journal of experimental education, 62(2), 127\u2013142.","journal-title":"The Journal of experimental education"},{"key":"128_CR59","doi-asserted-by":"crossref","first-page":"311","DOI":"10.1162\/tacl_a_00185","volume":"2","author":"R Passonneau","year":"2014","unstructured":"Passonneau, R., & Carpenter, B. (2014). The benefits of a model of annotation. Transactions of the Association for Computational Linguistics, 2, 311\u2013326.","journal-title":"Transactions of the Association for Computational Linguistics"},{"key":"128_CR60","unstructured":"Passonneau, R.J. (2006). Measuring agreement on set-valued items (masi) for semantic and pragmatic annotation. In Proceedings of the International Conference on Language Resources and Evaluation (LREC), pages 831\u2013836, Genoa, Italy."},{"key":"128_CR61","doi-asserted-by":"publisher","first-page":"107","DOI":"10.1017\/S1351324909005051","volume":"16","author":"RJ Passonneau","year":"2010","unstructured":"Passonneau, R.J. (2010). Formal and functional assessment of the pyramid method for summary content evaluation. Natural Language Engineering, 16, 107\u2013131.","journal-title":"Natural Language Engineering"},{"key":"128_CR62","unstructured":"Passonneau, R.J., Baker, C.F., Fellbaum, C., & Ide, N. (2012). The masc word sense corpus. In Proceedings of the Eight International Conference on Language Resources and Evaluation (LREC \u201912). European Language Resources Association (ELRA)."},{"key":"128_CR63","unstructured":"Passonneau, R.J., Chen, E., Guo, W., & Perin, D. (2013). Automated pyramid scoring of summaries using distributional semantics. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 143\u2013147, Sofia, Bulgaria. Association for Computational Linguistics."},{"key":"128_CR64","unstructured":"Passonneau, R.J., Goodkind, A., & Levy, E. (2007). Annotation of children\u2019s oral narrations: Modeling emergent narrative skills for computational applications. In Proceedings of the Twentieth Annual Meeting of the Florida Artificial Intelligence Research Society (FLAIRS-20), pages 253\u2013258."},{"key":"128_CR65","unstructured":"Passonneau, R.J., McKeown, K., & Sigelman, S. (2006). Applying the pyramid method in the 2006 Document Understanding Conference. In Proceedings of the 2006 Workshop of the Document Understanding Conference (DUC)."},{"key":"128_CR66","unstructured":"Passonneau, R.J., Nenkova, A., McKeown, K., & Sigelman, S. (2005). Applying the pyramid method in DUC 2005. In Proceedings of the 2005 Workshop of the Document Understanding Conference (DUC)."},{"issue":"2","key":"128_CR67","doi-asserted-by":"publisher","first-page":"8","DOI":"10.1080\/10790195.2013.10850365","volume":"43","author":"D Perin","year":"2013","unstructured":"Perin, D., Bork, R.H., Peverly, S.T., & Mason, L.H. (2013). A contextualized curricular supplement for developmental reading and writing. Journal of College Reading and Learning, 43(2), 8\u201338.","journal-title":"Journal of College Reading and Learning"},{"key":"128_CR68","volume-title":"The nation\u2019s report card: Writing 2002. Technical Report NCES 2003-529","author":"HR Persky","year":"2003","unstructured":"Persky, H.R., Daane, M.C., & Jin, Y. (2003). The nation\u2019s report card: Writing 2002. Technical Report NCES 2003-529. Washington: National Center for Education Statistics (NCES)."},{"key":"128_CR69","unstructured":"Pilehvar, M.T., Jurgens, D., & Navigli, R. (2013). Align, disambiguate and walk: A unified approach for measuring semantic similarity. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1341\u20131351, Sofia, Bulgaria. Association for Computational Linguistics."},{"issue":"2","key":"128_CR70","doi-asserted-by":"publisher","first-page":"136","DOI":"10.1111\/j.1467-9817.2010.01450.x","volume":"35","author":"A Proske","year":"2012","unstructured":"Proske, A., Narciss, S., & McNamara, D.S. (2012). Computer-based scaffolding to facilitate students\u2019 development of expertise in academic writing. Journal of Research in Reading, 35(2), 136\u2013152.","journal-title":"Journal of Research in Reading"},{"key":"128_CR71","unstructured":"Qazvinian, V., & Radev, D.R. (2012). A computational analysis of collective discourse. In Proceedings of the 2012 Conference on Collective Intelligence, Cambridge MA."},{"issue":"8","key":"128_CR72","first-page":"8","volume":"49","author":"BJ Reiser","year":"2012","unstructured":"Reiser, B.J., & Kenyon, L.K.B.L. (2012). Engaging students in the scientific practices of explanation and argumentation. Science and Children, 49(8), 8\u201313.","journal-title":"Science and Children"},{"key":"128_CR73","doi-asserted-by":"crossref","unstructured":"Roscoe, R.D., Allen, L.K., Weston, J.L., Crossley, S.A., & McNamara, D.S. (2015a). The Writing Pal intelligent tutoring system: Usability testing and development. Computers and Composition, 34, 39\u201359.","DOI":"10.1016\/j.compcom.2014.09.002"},{"issue":"4","key":"128_CR74","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1037\/a0032340","volume":"105","author":"RD Roscoe","year":"2013","unstructured":"Roscoe, R.D., & McNamara, D.S. (2013). Writing Pal: Feasibility of an intelligent writing strategy tutor in the high school classroom. Journal of Educational Psychology, 105(4), 1\u201316.","journal-title":"Journal of Educational Psychology"},{"key":"128_CR75","unstructured":"Roscoe, R.D., Snow, E.L., Allen, L.K., & McNamara, D.S. (2015b). Automated detection of essay revising patterns: applications for intelligent feedback in a writing tutor. Technology, Instruction, Cognition, and Learning, 10(1), 59\u201379."},{"issue":"4","key":"128_CR76","first-page":"325","volume":"15","author":"C Ros\u00e9","year":"2005","unstructured":"Ros\u00e9, C., & VanLehn, K. (2005). An evaluation of a hybrid language understanding approach for robust selection of tutoring goals. International Journal of Artificial Intelligence in Education, 15(4), 325.","journal-title":"International Journal of Artificial Intelligence in Education"},{"key":"128_CR77","unstructured":"Rudner, L.M., Garcia, V., & Welch, C. (2006). An evaluation of Intellimetric T M essay scoring system. The Journal of Technology Learning and Assessment, 4(4)."},{"key":"128_CR78","unstructured":"Saggion, H., Torres-Moreno, J.-M., da Cunha, I., SanJuan, E., & Vel\u00e1zquez-Morales, P. (2010). Multilingual summarization evaluation without human models. In Proceedings of Coling 2010, pages 1059\u20131067."},{"issue":"2-3","key":"128_CR79","doi-asserted-by":"publisher","first-page":"313","DOI":"10.1016\/S0166-218X(02)00205-6","volume":"126","author":"S Sakai","year":"2003","unstructured":"Sakai, S., Togasaki, M., & Yamazaki, K. (2003). A note on greedy algorithms for the maximum weighted independent set problem. Discrete Applied Mathematics, 126(2-3), 313\u2013322.","journal-title":"Discrete Applied Mathematics"},{"key":"128_CR80","volume-title":"The nation\u2019s report card: Writing 2007. Technical Report NCES 2008-468","author":"D Salahu-Din","year":"2008","unstructured":"Salahu-Din, D., Persky, H.R., & Miller, J. (2008). The nation\u2019s report card: Writing 2007. Technical Report NCES 2008-468. Washington: National Center for Education Statistics (NCES)."},{"issue":"5","key":"128_CR81","doi-asserted-by":"publisher","first-page":"643","DOI":"10.1002\/sce.21069","volume":"97","author":"V Sampson","year":"2013","unstructured":"Sampson, V., Enderle, P., Grooms, J., & Witte, S. (2013). Writing to learn by learning to write during the school science laboratory: Helping middle and high school students develop argumentative writing skills as they learn core ideas. Science Education, 97(5), 643\u2013670.","journal-title":"Science Education"},{"key":"128_CR82","doi-asserted-by":"crossref","DOI":"10.4324\/9780203122761","volume-title":"Handbook of Automated Essay Evaluation: Current Applications and Future Directions","author":"M Shermis","year":"2013","unstructured":"Shermis, M., & Burstein, J. (2013). Handbook of Automated Essay Evaluation: Current Applications and Future Directions. New York: New York: Routledge."},{"key":"128_CR83","volume-title":"WISE Science","author":"JD Slotta","year":"2009","unstructured":"Slotta, J.D., & Linn, M.C. (2009). WISE Science. New York: Teachers College Press."},{"key":"128_CR84","volume-title":"The Wisdom of Crowds","author":"J Surowiecki","year":"2004","unstructured":"Surowiecki, J. (2004). The Wisdom of Crowds. New York: Doubleday."},{"key":"128_CR85","unstructured":"Teufel, S., & van Halteren, H. (2004). Evaluating information content by factoid analysis: Human annotation and stability. In Lin, D., & Wu, D. (Eds.) Proceedings of EMNLP 2004, pages 419\u2013426, Barcelona, Spain. Association for Computational Linguistics."},{"key":"128_CR86","volume-title":"The propositional analysis system. Technical Report 87-2, University of Colorado","author":"AA Turner","year":"1987","unstructured":"Turner, A.A. (1987). The propositional analysis system. Technical Report 87-2, University of Colorado. Boulder: Department of Psychology and Institute of Cognitive Science."},{"key":"128_CR87","volume-title":"The construction and use of a propositional analysis system. Technical Report JSAS Catalog of Selected Documents in Psychology, no. 1713","author":"AA Turner","year":"1978","unstructured":"Turner, A.A., & Greene, E. (1978). The construction and use of a propositional analysis system. Technical Report JSAS Catalog of Selected Documents in Psychology, no. 1713. Washington, DC: American Psychological Association."},{"key":"128_CR88","unstructured":"van Dijk, T.A., & Kintsch, W. (1977). Cognitive psychology and discourse: Recalling and summarizing stories. In Dressier, W.U. (Ed.) Trends in text-linguistics, pages 61\u201380. De Gruyter, New York."},{"key":"128_CR89","unstructured":"van Dijk, T.A., & Kintsch, W. (1983). Strategies of Discourse Comprehension. New York: Academic Press."},{"key":"128_CR90","unstructured":"VanLehn, K., Jordan, P., & Ros\u00e9, C.P. (2002). The architecture of Why2-Atlas: a coach for qualitative physics essay writing. In Cerri, S.A., Gouarderes, G., & Paraguacu, F. (Eds.) Intelligent Tutoring Systems, 2002: 6th International Conference, pages 158\u2013167, Berlin. Springer."},{"issue":"4","key":"128_CR91","doi-asserted-by":"publisher","first-page":"275","DOI":"10.1097\/TLD.0b013e3181ff5a88","volume":"30","author":"C Westby","year":"2010","unstructured":"Westby, C., Culatta, B., Lawrence, B., & Hall-Kenyon, K. (2010). Summarizing expository texts. Topics in Language Disorders, 30(4), 275\u2013287.","journal-title":"Topics in Language Disorders"},{"key":"128_CR92","unstructured":"Wiemer-Hastings, P., Wiemer-Hastings, K., & Graesser, A.C. (1999). Improving an intelligent tutor\u2019s comprehension of students with latent semantic analysis. In Lajoie, S.P., & Vivet, M. (Eds.) Artificial Intelligence in Education, pages 535\u2013542. IOS Press, Amsterdam."},{"key":"128_CR93","doi-asserted-by":"publisher","first-page":"63","DOI":"10.1002\/(SICI)1099-0720(199611)10:7<63::AID-ACP438>3.0.CO;2-5","volume":"10","author":"J Wiley","year":"1996","unstructured":"Wiley, J., & Voss, J.F. (1996). The effects of playing historian on learning in history. Applied Cognitive Psychology, 10, 63\u201372.","journal-title":"Applied Cognitive Psychology"},{"key":"128_CR94","doi-asserted-by":"crossref","unstructured":"Yang, Q., Passonneau, R.J., & de Melo, G. (2016). PEAK: Pyramid evaluation via automated knowledge extraction. In Proceedings of the 30th AAAI Conference on Artificial Intelligence (AAAI 2016). AAAI Press.","DOI":"10.1609\/aaai.v30i1.10336"},{"issue":"4","key":"128_CR95","doi-asserted-by":"publisher","first-page":"338","DOI":"10.1002\/tea.20008","volume":"41","author":"LD Yore","year":"2004","unstructured":"Yore, L.D., Hand, B.M., & Florence, M.K. (2004). Scientists\u2019 views of science, models of writing, and science writing practices. Journal of Research in Science Teaching, 41(4), 338\u2013369.","journal-title":"Journal of Research in Science Teaching"},{"key":"128_CR96","volume-title":"Human Behaviour and the Principle of Least Effort","author":"GK Zipf","year":"1949","unstructured":"Zipf, G.K. (1949). Human Behaviour and the Principle of Least Effort. Cambridge: Addison-Wesley."}],"container-title":["International Journal of Artificial Intelligence in Education"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/link.springer.com\/article\/10.1007\/s40593-016-0128-6\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"http:\/\/link.springer.com\/content\/pdf\/10.1007\/s40593-016-0128-6.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"http:\/\/link.springer.com\/content\/pdf\/10.1007\/s40593-016-0128-6.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,3,4]],"date-time":"2026-03-04T18:11:59Z","timestamp":1772647919000},"score":1,"resource":{"primary":{"URL":"http:\/\/link.springer.com\/10.1007\/s40593-016-0128-6"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2016,12,5]]},"references-count":96,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2018,3]]}},"alternative-id":["128"],"URL":"https:\/\/doi.org\/10.1007\/s40593-016-0128-6","relation":{},"ISSN":["1560-4292","1560-4306"],"issn-type":[{"value":"1560-4292","type":"print"},{"value":"1560-4306","type":"electronic"}],"subject":[],"published":{"date-parts":[[2016,12,5]]},"assertion":[{"value":"5 December 2016","order":1,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"This content has been made available to all.","name":"free","label":"Free to read"}]}}