{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,1]],"date-time":"2026-04-01T18:28:36Z","timestamp":1775068116325,"version":"3.50.1"},"reference-count":176,"publisher":"Elsevier BV","issue":"1","license":[{"start":{"date-parts":[[2014,10,23]],"date-time":"2014-10-23T00:00:00Z","timestamp":1414022400000},"content-version":"tdm","delay-in-days":0,"URL":"http:\/\/www.springer.com\/tdm"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Int J Artif Intell Educ"],"published-print":{"date-parts":[[2015,3]]},"DOI":"10.1007\/s40593-014-0026-8","type":"journal-article","created":{"date-parts":[[2014,10,22]],"date-time":"2014-10-22T10:33:40Z","timestamp":1413974020000},"page":"60-117","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":293,"title":["The Eras and Trends of Automatic Short Answer Grading"],"prefix":"10.1016","volume":"25","author":[{"given":"Steven","family":"Burrows","sequence":"first","affiliation":[]},{"given":"Iryna","family":"Gurevych","sequence":"additional","affiliation":[]},{"given":"Benno","family":"Stein","sequence":"additional","affiliation":[]}],"member":"78","published-online":{"date-parts":[[2014,10,23]]},"reference":[{"key":"26_CR1","unstructured":"Aleven, V., Ogan, A., Popescu, O., Torrey, C., Koedinger, K. (2004). Evaluating the effectiveness of a tutorial dialogue system for self-explanation. J.C. Lester, R.M. Vicari, F. Paraguacu (Eds.), Proceedings of the 7th international conference on intelligent tutoring systems volume 3220 of lecture notes in computer science (pp. 443\u2013454). Maceio: Springer."},{"key":"26_CR2","unstructured":"Alfonseca, E., & P\u00e9rez, D. (2004). Automatic assessment of open ended questions with a BLEU-inspired algorithm and shallow NLP. In J. Vicedo, P. Mart\u00ednez-Barco, Mu\u0144oz, M. Saiz Noeda (Eds.), Advances in natural language processing, volume 3230 of lecture notes in computer science (pp. 25\u201335). Berlin: Springer."},{"issue":"3","key":"26_CR3","first-page":"53","volume":"8","author":"E Alfonseca","year":"2005","unstructured":"Alfonseca, E., Carro, R.M., Freire, M., Ortigosa, A., P\u00e9rez, D., Rodriguez, P. (2005). Authoring of adaptive computer assisted assessment of free-text answers. Educational Technology & Society, 8(3), 53\u201365.","journal-title":"Educational Technology & Society"},{"issue":"3","key":"26_CR4","first-page":"1","volume":"4","author":"Y Attali","year":"2006","unstructured":"Attali, Y., & Burstein, J. (2006). Automated essay Scoring with e-rater V.2. The Journal of Technology, Learning, and Assessment, 4(3), 1\u201331.","journal-title":"The Journal of Technology, Learning, and Assessment"},{"key":"26_CR5","unstructured":"Attali, Y., Powers, D., Freedman, M., Harrison, M., Obetz, S. (2008). Automated scoring of short-answer open-ended GRE subject test items. Technical Report RR-08-20. Princeton: Educational Testing Service."},{"key":"26_CR6","unstructured":"Bachman, L.F., Carr, N., Kamei, G., Kim, M., Pan, M.J., Salvador, C., Sawaki, Y. (2002). A reliable approach to automatic assessment of short answer free responses. In S.C. Tseng, T.E. Chen, Y.F. Liu (Eds.), Proceedings of the 19th international conference on computational linguistics, volume 2 of COLING \u201902 (pp. 1\u20134). Taipei: Association for Computational Linguistics."},{"key":"26_CR7","unstructured":"Bailey, S. (2008). Content assessment in intelligent computer-aided language learning: meaning error diagnosis for english as a second language. Ph.D. thesis, Columbus: Ohio State University."},{"key":"26_CR8","unstructured":"Bailey, S., & Meurers, D. (2008). Diagnosing meaning errors in short answers to reading comprehension questions. In J. Tetreault, J. Burstein, R. De Felice (Eds.), Proceedings of the 3rd ACL workshop on innovative use of NLP for building educational applications (pp. 107\u2013115). Columbus: Association for Computational Linguistics."},{"issue":"4","key":"26_CR9","doi-asserted-by":"crossref","first-page":"357","DOI":"10.1037\/1082-989X.2.4.357","volume":"4","author":"R Bakeman","year":"1997","unstructured":"Bakeman, R., McArthur, D., Quera, V., Robinson, B.F. (1997). Detecting sequential patterns and determining their reliability with fallible observers. Psychological Methods, 4(4), 357\u2013370.","journal-title":"Psychological Methods"},{"key":"26_CR10","unstructured":"B\u00e4r, D., Zesch, T., Gurevych, I. (2011). A reflective view on text similarity. In R. Mitkov & G. Angelova (Eds.), Proceedings of the international conference on recent advances in natural language processing (pp. 515\u2013520). Hissar: Association for Computational Linguistics."},{"key":"26_CR11","unstructured":"B\u00e4r, D., Zesch, T., Gurevych, I. (2012a). Text reuse detection using a composition of text similarity measures. In P. Bhattacharrya, R. Sangal, M. Kay, C. Boitet (Eds.), Proceedings of the 24th international conference on computational linguistics volume 1 of COLING \u201912 (pp. 167\u2013184. Mumbai: Indian Institute of Technology Bombay."},{"key":"26_CR12","unstructured":"B\u00e4r, D., Biemann, C., Gurevych, I., Zesch, T. (2012b). UKP: computing semantic textual similarity by combining multiple content similarity measures. In S. Manandhar, D. Yuret (Eds.), Proceedings of the 6th international workshop on semantic evaluation (pp. 435\u2013440). Montreal: Association for Computational Linguistics."},{"key":"26_CR13","unstructured":"B\u00e4r, D., Zesch, T., Gurevych, I. (2013). DKPro similarity: an open source framework for text similarity. In H. Schuetze, P. Fung, M. Poesio (Eds.), Proceedings of the 51st annual meeting of the association for computational linguistics. System demonstrations (pp. 121\u2013126). Sofia: Association for Computational Linguistics."},{"key":"26_CR14","unstructured":"Bar-Haim, R., Dagan, I., Dolan, B., Ferro, L., Giampiccolo, D., Magnini, B., Szpektor, I. (2006). The second pascal recognising textual entailment challenge. In B. Magnini & I. Dagan (Eds.), Proceedings of the 2nd Pascal challenges workshop on recognising textual entailment (pp. 1\u20139). Venice."},{"issue":"3","key":"26_CR15","doi-asserted-by":"crossref","first-page":"319","DOI":"10.1080\/0969594X.2011.555329","volume":"18","author":"II Bejar","year":"2011","unstructured":"Bejar, I.I. (2011). A validity-based approach to quality control and assurance of automated scoring. Assessment in Education: Principles, Policy & Practice, 18(3), 319\u2013341.","journal-title":"Assessment in Education: Principles, Policy & Practice"},{"key":"26_CR16","unstructured":"Bennett, R.E. (2011). Automated scoring of constructed-response literacy and mathematics items. White paper, Educational Testing Service. Princeton."},{"key":"26_CR17","unstructured":"Bentivogli, L., Dagan, I., Dang, H.T., Giampiccolo, D., Magnini, B. (2009). The fifth PASCAL recognizing textual entailment challenge. In Proceedings of the 2nd text analysis conference (pp. 1\u201315). Gaithersburg: National Institute of Standards and Technology."},{"key":"26_CR18","unstructured":"Bentivogli, L., Clark, P., Dagan, I., Giampiccolo, D. (2010). The sixth PASCAL recognizing textual entailment challenge. In Proceedings of the 3rd text analysis conference (pp. 1\u201318). Gaithersburg: National Institute of Standards and Technology."},{"key":"26_CR19","unstructured":"Bentivogli, L., Clark, P., Dagan, I., Giampiccolo, D. (2011). The seventh PASCAL recognizing textual entailment challenge. In Proceedings of the 4th text analysis conference (pp. 1\u201316). Gaithersburg: National Institute of Standards and Technology."},{"issue":"2","key":"26_CR20","first-page":"123","volume":"24","author":"L Breiman","year":"1996","unstructured":"Breiman, L. (1996). Bagging predictors. Machine Learning, 24(2), 123\u2013140.","journal-title":"Machine Learning"},{"key":"26_CR21","unstructured":"Bukai, O., Pokorny, R., Haynes, J. (2006). An automated short-free-text scoring system: development and assessment. In Proceedings of the 20th interservice\/industry training, simulation, and education conference (pp. 1\u201311). National Training and Simulation Association."},{"key":"26_CR22","unstructured":"Burrows, S., & D\u2019Souza, D. (2005). Management of teaching in a complex setting. In J. Hurst & J. Sheard (Eds.), Proceedings of the 2nd melbourne computing education conventicle (pp. 1\u20138). Melbourne."},{"issue":"2","key":"26_CR23","first-page":"151","volume":"37","author":"S Burrows","year":"2007","unstructured":"Burrows, S., Tahaghoghi, S.M.M., Zobel, J. (2007). Efficient plagiarism detection for large code repositories. Software: Practice and Experience, 37(2), 151\u2013175.","journal-title":"Software: Practice and Experience"},{"issue":"3","key":"26_CR24","doi-asserted-by":"crossref","first-page":"43:1","DOI":"10.1145\/2483669.2483676","volume":"4","author":"S Burrows","year":"2013","unstructured":"Burrows, S., Potthast, M., Stein, B. (2013). Paraphrase acquisition via crowdsourcing and machine learning. ACM Transactions on Intelligent Systems and Technology, 4(3), 43:1\u201343:21.","journal-title":"ACM Transactions on Intelligent Systems and Technology"},{"issue":"1","key":"26_CR25","first-page":"1","volume":"44","author":"S Burrows","year":"2014","unstructured":"Burrows, S., Uitdenbogerd, A.L., Turpin, A. (2014). Comparing techniques for authorship attribution of source code. Software: Practice and Experience, 44(1), 1\u201332.","journal-title":"Software: Practice and Experience"},{"key":"26_CR26","unstructured":"Burstein, J., Kaplan, R., Wolff, S., Lu, C. (1996). Using lexical semantic techniques to classify free-responses. In E. Viegas (Ed.), Proceedings of the ACL SIGLEX workshop on breadth and depth of semantic lexicons (pp. 20\u201329). Santa Cruz: Association for Computational Linguistics."},{"issue":"2","key":"26_CR27","doi-asserted-by":"crossref","first-page":"489","DOI":"10.1016\/j.compedu.2010.02.012","volume":"55","author":"PG Butcher","year":"2010","unstructured":"Butcher, P.G., & Jordan, S.E. (2010). A comparison of human and computer marking of short free-text student responses. Computers & Education, 55(2), 489\u2013499.","journal-title":"Computers & Education"},{"key":"26_CR28","unstructured":"Callear, D., Jerrams-Smith, J., Soh, V. (2001). CAA of short non-MCQ answers. In M. Danson & C. Eabry (Eds.), Proceedings of the 5th computer assisted assessment conference (pp. 1\u201314). Loughborough: Loughborough University."},{"issue":"1","key":"26_CR29","doi-asserted-by":"crossref","first-page":"37","DOI":"10.1177\/001316446002000104","volume":"20","author":"J Cohen","year":"1960","unstructured":"Cohen, J. (1960). A coefficient of agreement for nominal scales. Educational and Psychological Measurement, 20(1), 37\u201346.","journal-title":"Educational and Psychological Measurement"},{"issue":"4","key":"26_CR30","doi-asserted-by":"crossref","first-page":"213","DOI":"10.1037\/h0026256","volume":"70","author":"J Cohen","year":"1968","unstructured":"Cohen, J. (1968). Weighted kappa: nominal scale agreement with provision for scaled disagreement of partial credit. Psychological Bulletin, 70(4), 213\u2013220.","journal-title":"Psychological Bulletin"},{"issue":"1","key":"26_CR31","doi-asserted-by":"crossref","first-page":"155","DOI":"10.1037\/0033-2909.112.1.155","volume":"112","author":"J Cohen","year":"1992","unstructured":"Cohen, J. (1992). A power primer. Psychological Bulletin, 112(1), 155\u2013159.","journal-title":"Psychological Bulletin"},{"issue":"1","key":"26_CR32","doi-asserted-by":"crossref","first-page":"17","DOI":"10.1080\/0968776042000339772","volume":"13","author":"G Conole","year":"2005","unstructured":"Conole, G., & Warburton, B. (2005). A review of computer-assisted assessment. Journal of the Association for Learning Technology, 13(1), 17\u201331.","journal-title":"Journal of the Association for Learning Technology"},{"key":"26_CR33","unstructured":"Conort, X. (2012). Short answer scoring \u2013 explanation of Gxav solution. ASAP \u201912 SAS Methodology paper, Gear Analytics."},{"key":"26_CR34","unstructured":"Cowie, J., & Wilks, Y. (2000). Information extraction. In R. Dale, H. Moisl, H. Somers (Eds.), Handbook of natural language processing, 1st edn. (Chap. 10, pp. 241\u2013260). New York City: Marcel Dekker."},{"key":"26_CR35","unstructured":"Csink, L., Gy\u00f6rgy, A., Raincs\u00e1k, Z., Schmuck, B., Sima, D., Sziklai, Z., Sz\u00f6ll\u00f6si, S. (2003). Intelligent assessment systems for e-learning. In F. Udrescu, I.G. Rosca , G.M. Sandulescu, R. Stroe (Eds.), Proceedings of the 4th European conference on e-commerce, e-learning, e-business, e-work, e-health, e-banking, e-democracy, e-government, BB & on-line services applications, new working environments, virtual institutes, and their influences on the economic and social environment E-COMM-LINE (pp. 224\u2013229). Bucharest: R&D Institute for Automation Bucharest and Academy of Economic Studies Bucharest."},{"key":"26_CR36","unstructured":"Cutrone, L., & Chang, M. (2010). Automarking: automatic assessment of open questions. In M. Jemni, D. Sampson, Kinshuk, J.M. Spector (Eds.), Proceedings of the 10th IEEE international conference on advanced learning technologies (pp. 143\u2013147). Sousse: IEEE."},{"key":"26_CR37","unstructured":"Cutrone, L., Chang, M., Kinshuk (2011). Auto-assessor: computerized assessment system for marking student\u2019s short-answers automatically. In: N.S. Narayanaswamy, M.S. Krishnan, Kinshuk, R. Srinivasan (Eds.), Proceedings of the 3rd IEEE international conference on technology for education (pp. 81\u201388). Chennai: IEEE."},{"key":"26_CR38","doi-asserted-by":"crossref","unstructured":"Dagan, I., Glickman, O., Gan, R., Magnini, B. (2006). The PASCAL recognising textual entailment challenge. In J. Qui\u00f1onero-Candela, I. Dagan, B. Magnini, F. D\u2019Alch\u00e9 Buc (Eds.), Machine learning challenges, volume 3944 of lecture notes in computer science (pp. 177\u2013190). Springer.","DOI":"10.1007\/11736790_9"},{"key":"26_CR39","unstructured":"Dorr, B., Hendler, J., Blanksteen, S., Migdaloff, B. (1995). On beyond syntax: use of lexical conceptual structure for intelligent tutoring. In V.M. Holland, J. Kaplan, M. Sams (Eds.), Intelligent language tutors, 1st edn. (pp. 289\u2013311). Mahwah: Lawrence Erlbaum Publishers."},{"key":"26_CR40","unstructured":"Dzikovska, M.O., Nielsen, R.D., Brew, C. (2012). Towards effective tutorial feedback for explanation questions: a dataset and baselines. In J. Chu-Carroll, E. Fosler-Lussier, E. Riloff, S. Bangalore (Eds.), Proceedings of the 12th conference of the north american chapter of the association for computational linguistics: human language technologies (pp. 200\u2013210). Montreal: Association for Computational Linguistics."},{"key":"26_CR41","unstructured":"Dzikovska, M.O., Nielsen, R.D., Brew, C., Leacock, C., Giampiccolo, D., Bentivogli, L., Clark, P., Dagan, I., Dang, H.T. (2013). SemEval-2013 task 7: the joint student response analysis and eighth recognizing textual entailment challenge. In M. Diab, T. Baldwin, M. Baroni (Eds.), Proceedings of the 2nd joint conference on lexical and computational semantics (pp. 1\u201312). Atlanta."},{"key":"26_CR42","unstructured":"Evens, M.W., Brandle, S., Chang, R.C., Freedman, R., Glass, M., Lee, Y.H., Shim, L.S., Woo, C.W., Zhang, Y., Zhou, Y., Michael, J.A., Rovick, A.A. (2001). CIRCSIM-Tutor: an intelligent tutoring system using natural language dialogue, In Proceedings of the 12th midwest artificial intelligence and cognitive science conference (pp. 16\u201323). Oxford."},{"key":"26_CR43","doi-asserted-by":"crossref","unstructured":"Fleiss, J.L. (2003). The measurement of interrater agreement. In J.L. Fleiss, B. Levin, M.C. Paik (Eds.), Statistical methods for rates and proportions, 3rd edn. (Chap. 18, pp. 598\u2013626). Wiley.","DOI":"10.1002\/0471445428.ch18"},{"key":"26_CR44","unstructured":"Gabrilovich, E., & Markovitch, S. (2006). Overcoming the brittleness bottleneck using wikipedia: enhancing text categorization with encyclopedic knowledge. In A. Cohn (Ed.), Proceedings of the 21st national conference on artificial intelligence (Vol. 2, pp. 1301\u20131306). Boston: AAAI Press."},{"issue":"1","key":"26_CR45","doi-asserted-by":"crossref","first-page":"45","DOI":"10.1111\/j.1745-3984.1980.tb00813.x","volume":"17","author":"LR Gay","year":"1980","unstructured":"Gay, L.R. (1980). The comparative effects of multiple-choice versus short-answer tests on retention. Journal of Educational Measurement, 17(1), 45\u201350.","journal-title":"Journal of Educational Measurement"},{"key":"26_CR46","unstructured":"Giampiccolo, D., Magnini, B., Sommarive, V., Gan, R., Dolan, B. (2007). The third PASCAL recognizing textual entailment challenge. In S. Sekine, K. Inui, I. Dagan, B. Dolan, D. Giampiccolo, B. Magnini (Eds.), Proceedings of the ACL-PASCAL workshop on textual entailment and paraphrasing (pp. 1\u20139). Prague: Association for Computational Linguistics."},{"key":"26_CR47","unstructured":"Giampiccolo, D., Dang, H.T., Magnini, B., Dagan, I., Cabrio, E., Dolan, B. (2008). The fourth PASCAL recognizing textual entailment challenge. In Proceedings of the 1st text analysis conference (pp. 1\u201311). Gaithersburg: National Institute of Standards and Technology."},{"key":"26_CR48","unstructured":"Gollub, T., Burrows, S., Stein, B. (2012a). First experiences with TIRA for reproducible evaluation in information retrieval. In A. Trotman, C.L.A. Clarke, I. Ounis, J.S. Culpepper, M.A. Cartright, S. Geva (Eds.), Proceedings of the 1st SIGIR workshop on open source information retrieval (pp. 52\u201355). Portland: University of Otago."},{"key":"26_CR49","doi-asserted-by":"crossref","unstructured":"Gollub, T., Stein, B., Burrows, S. (2012b). Ousting ivory tower research: towards a web framework for providing experiments as a service. In B. Hersh, J. Callan, Y. Maarek, M. Sanderson (Eds.), Proceedings of the 35th international ACM SIGIR conference on research and development in information retrieval (pp. 1125\u20131126). Portland: ACM.","DOI":"10.1145\/2348283.2348501"},{"key":"26_CR50","doi-asserted-by":"crossref","unstructured":"Gollub, T., Stein, B., Burrows, S., Hoppe, D. (2012c). TIRA: configuring, executing, and disseminating information retrieval experiments. In A.M. Tjoa, S. Liddle, K.D. Schewe, X. Zhou (Eds.), Proceedings of the 9th international workshop on text-based information retrieval at DEXA (pp. 151\u2013155). Vienna: IEEE.","DOI":"10.1109\/DEXA.2012.55"},{"key":"26_CR51","unstructured":"Gonzalez-Barbone, V., & Llamas-Nistal, M. (2008). eAssessment of open questions: an educator\u2019s perspective. In C. Traver, M. Ohland, J. Prey, T. Mitchell (Eds.), Proceedings of the 38th annual frontiers in education conference (pp. F2B\u20131\u2013F2B\u20136). Saratoga Springs: IEEE."},{"issue":"4","key":"26_CR52","doi-asserted-by":"crossref","first-page":"612","DOI":"10.1109\/TE.2005.856149","volume":"48","author":"AC Graesser","year":"2005","unstructured":"Graesser, A.C., Chipman, P., Haynes, B.C., Olney, A. (2005). AutoTutor: an intelligent tutoring system with mixed-initiative dialogue. IEEE Transactions on Education, 48(4), 612\u2013618.","journal-title":"IEEE Transactions on Education"},{"key":"26_CR53","unstructured":"G\u00fctl, C. (2007). e-Examiner: towards a fully-automatic knowledge assessment tool applicable in adaptive e-learning systems. In P.H. Ghassib (Ed.), Proceedings of the 2nd international conference on interactive mobile and computer aided learning (pp. 1\u201310). Amman."},{"issue":"1","key":"26_CR54","first-page":"1","volume":"3","author":"C G\u00fctl","year":"2008","unstructured":"G\u00fctl, C. (2008). Moving towards a fully automatic knowledge assessment tool. International Journal of Emerging Technologies in Learning, 3(1), 1\u201311.","journal-title":"International Journal of Emerging Technologies in Learning"},{"key":"26_CR55","unstructured":"Gy\u00f6rgy, A., & Vajda, I. (2007). Intelligent mathematics assessment in eMax. In Proceedings of the 8th Africon conference (pp. 1\u20136). Windhoek: IEEE."},{"key":"26_CR56","unstructured":"Hahn, M., & Meurers, D. (2012). Evaluating the meaning of answers to reading comprehension questions: a semantics-based approach. In J. Tetreault, J. Burstein, C. Leacock (Eds.), Proceedings of the 7th workshop on building educational applications using NLP (pp. 326\u2013336). Montreal: Association for Computational Linguistics."},{"key":"26_CR57","unstructured":"Haley, D.T., Thomas, P., Roeck, A.D., Petre, M. (2007). Measuring improvement in latent semantic analysis-based marking systems: using a computer to mark questions about HTML. In S. Mann & Simon (Eds.), Proceedings of the 9th australasian conference on computing education, volume 66 of ACE (pp. 35\u201342). Ballarat: Australian Computer Society."},{"issue":"1","key":"26_CR58","doi-asserted-by":"crossref","first-page":"10","DOI":"10.1145\/1656274.1656278","volume":"11","author":"M Hall","year":"2009","unstructured":"Hall, M., Frank, E., Holmes, G., Pfahringer, B., Reutemann, P., Witten, I.H. (2009). The WEKA data mining software: an update. SIGKDD Explorations, 11(1), 10\u201318.","journal-title":"SIGKDD Explorations"},{"key":"26_CR59","unstructured":"Hamp, B., & Feldweg, H. (1997). GermaNet\u2014a lexical-semantic net for german. In P. Vossen, G. Adriaens, N. Calzolari, A. Sanfilippo, Y. Wilks (Eds.), Proceedings of the 1st ACL workshop on information extraction and building of lexical semantic resources for NLP applications (pp. 9\u201316). Madrid: Association for Computational Linguistics."},{"key":"26_CR60","unstructured":"Hayashi, F. (2000). Finite-sample properties of OLS. In Econometrics (Chap. 1, pp. 3\u201387). Princeton: Princeton University Press."},{"key":"26_CR61","unstructured":"Heilman, M., & Madnani, N. (2013). ETS: domain adaptation and stacking for short answer scoring. In M. Diab, T. Baldwin, M. Baroni (Eds.), Proceedings of the 2nd joint conference on lexical and computational semantics (Vol. 2, pp. 275\u2013279). Atlanta."},{"key":"26_CR62","unstructured":"Hewlett Foundation (2012). Automated student assessment prize: phase two \u2013 short answer scoring. Kaggle Competition."},{"issue":"5","key":"26_CR63","first-page":"31","volume":"15","author":"L Hirschman","year":"2000","unstructured":"Hirschman, L., Breck, E., Light, M., Burger, J.D., Ferro, L. (2000). Automated grading of short-answer tests. Intelligent Systems and their Applications, 15(5), 31\u201335.","journal-title":"Intelligent Systems and their Applications"},{"key":"26_CR64","unstructured":"Hirst, G., & St-Onge, D. (1998). Lexical chains as representations of contexts for the detection and correction of malapropisms. In C. Fellbaum (Ed.), WordNet: an electronic lexical database, language, speech, and communication (Chap. 13, pp. 305\u2013332). MIT Press."},{"key":"26_CR65","unstructured":"Horbach, A., Palmer, A., Pinkal, M. (2013). Using the text to evaluate short answers for reading comprehension exercises. In M. Diab, T. Baldwin, M. Baroni (Eds.), Proceedings of the 2nd joint conference on lexical and computational semantics (Vol. 1, pp. 286\u2013295). Atlanta: Association for Computational Linguistics."},{"issue":"2","key":"26_CR66","doi-asserted-by":"crossref","first-page":"327","DOI":"10.1142\/S0218213011000188","volume":"20","author":"WJ Hou","year":"2011","unstructured":"Hou, W.J., & Tsao, J.H. (2011). Automatic assessment of students\u2019 free-text answers with different levels. International Journal on Artificial Intelligence Tools, 20(2), 327\u2013347.","journal-title":"International Journal on Artificial Intelligence Tools"},{"key":"26_CR67","unstructured":"Hou, W.J., Tsao, J.H., Li, S.Y., Chen, L. (2010). Automatic assessment of students\u2019 free-text answers with support vector machines. In M. Ali, C. Fyfe, N. Garc\u00eda-Pedrajas, F. Herrera (Eds.), Proceedings of the 23rd international conference on industrial engineering and other applications of applied intelligent systems (Vol. 1, pp. 235\u2013243). Cordoba: Springer."},{"key":"26_CR68","unstructured":"Hou, W.-J., Tsao, J.-H., Lu, C.-S., Chen, L.-C. (2011). Free-text assessment of students\u2019 answers with different feature selections. In K. Tang & H. Koguchi (Eds.), Proceedings of the 5th international conference on e-commerce, e-administration, e-society, e-education, and e-technology (pp. 2489\u20132510). Tokyo: Knowledge Association of Taiwan and International Business Academics Consortium."},{"key":"26_CR69","unstructured":"Hou, W.-J., Lu, C.-S., Chang, C.-P., Chen, H.-Y. (2012). Learning diagnosis for students\u2019 free-text answers with feature-based approaches. In Proceedings of the 1st international conference on information and computer applications (Vol. 24, pp. 42\u201347). Hong Kong: International Association of Computer Science and Information Technology."},{"key":"26_CR70","unstructured":"Intelligent Assessment Technologies (2009). E-assessment of short-answer questions. White paper. Coatbridge."},{"key":"26_CR71","unstructured":"Jiang, J.J., & Conrath, D.W. (1997). Semantic Similarity based on corpus statistics and lexical taxonomy. In L.-S. Lee, K.-J. Chen, C.-R. Huang, R. Sproat (Eds.), Proceedings of the 10th international conference on research in computational linguistics (pp. 1\u201315). Taipei."},{"key":"26_CR72","unstructured":"Jimenez, S., Becerra, C., Universitaria, C., Gelbukh, A. (2013). SOFTCARDINALITY: hierarchical text overlap for student response analysis. In M. Diab, T. Baldwin, M. Baroni (Eds.), Proceedings of the 2nd joint conference on lexical and computational semantics (Vol. 2, pp. 280\u2013284). Atlanta."},{"key":"26_CR73","unstructured":"Jordan, S. (2007). Computer based assessment with short free responses and tailored feedback. In P. Chin, K. Clark, S. Doyle, P. Goodhew, T. Madden, S. Meskin, T. Overton, J. Wilson (Eds.), Proceedings of the 2nd science learning and teaching conference (pp. 158\u2013163). Keele."},{"key":"26_CR74","doi-asserted-by":"crossref","first-page":"17","DOI":"10.11120\/ndir.2008.00040017","volume":"4","author":"S Jordan","year":"2008","unstructured":"Jordan, S. (2008). Online interactive assessment with short free-text questions and tailored feedback. New Directions, 4, 17\u201320.","journal-title":"New Directions"},{"issue":"1","key":"26_CR75","first-page":"11","volume":"3","author":"S Jordan","year":"2009","unstructured":"Jordan, S. (2009a). Assessment for learning: pushing the boundaries of computer-based assessment. Practitioner Research in Higher Education, 3(1), 11\u201319.","journal-title":"Practitioner Research in Higher Education"},{"key":"26_CR76","unstructured":"Jordan, S. (2009b). Investigating the use of short free text questions in online assessment. Final project report, Centre for the Open Learning of Mathematics, Science, Computing and Technology, The Open University, Milton Keynes."},{"key":"26_CR77","unstructured":"Jordan, S. (2012a). Short-answer e-assessment questions: five years on. In D. Whitelock, G. Wills, B. Warburton (Eds.), Proceedings of the 15th international computer assisted assessment conference (pp. 1\u20131). Southampton."},{"issue":"2","key":"26_CR78","doi-asserted-by":"crossref","first-page":"818","DOI":"10.1016\/j.compedu.2011.10.007","volume":"58","author":"S Jordan","year":"2012","unstructured":"Jordan, S. (2012b). Student engagement with assessment and feedback: some lessons from short-answer free-text e-assessment questions. Computers & Education, 58(2), 818\u2013834.","journal-title":"Computers & Education"},{"issue":"2","key":"26_CR79","doi-asserted-by":"crossref","first-page":"371","DOI":"10.1111\/j.1467-8535.2008.00928.x","volume":"40","author":"S Jordan","year":"2009","unstructured":"Jordan, S., & Mitchell, T. (2009). e-Assessment for learning? The potential of short-answer free-text questions with tailored feedback. British Journal of Educational Technology, 40(2), 371\u2013385.","journal-title":"British Journal of Educational Technology"},{"key":"26_CR80","unstructured":"Jordan, S., Brockbank, B., Butcher, P. (2007). Extending the pedagogic role of online interactive assessment: providing feedback on short free-text responses. In Proceedings of the international online conference on assessment design for learner responsibility."},{"issue":"1\u20132","key":"26_CR81","doi-asserted-by":"crossref","first-page":"81","DOI":"10.1093\/biomet\/30.1-2.81","volume":"30","author":"MG Kendall","year":"1938","unstructured":"Kendall, M.G. (1938). A new measure of rank correlation. Biometrika, 30(1\u20132), 81\u201393.","journal-title":"Biometrika"},{"key":"26_CR82","unstructured":"Klein, R., Kyrilov, A., Tokman, M. (2011). Automated assessment of short free-text responses in computer science using latent semantic analysis. In G. R\u00f6\u00dfling, T. Naps, C. Spannagel (Eds.), Proceedings of the 16th annual joint conference on innovation and technology in computer science education (pp. 158\u2013162). Darmstadt: ACM."},{"issue":"4","key":"26_CR83","doi-asserted-by":"crossref","first-page":"212","DOI":"10.1207\/s15430421tip4104_2","volume":"41","author":"DR Krathwohl","year":"2002","unstructured":"Krathwohl, D.R. (2002). A revision of bloom\u2019s taxonomy: an overview. Theory into Practice, 41(4), 212\u2013219.","journal-title":"Theory into Practice"},{"issue":"2\u20133","key":"26_CR84","doi-asserted-by":"crossref","first-page":"259","DOI":"10.1080\/01638539809545028","volume":"25","author":"TK Landauer","year":"1998","unstructured":"Landauer, T.K., Foltz, P.W., Laham, D. (1998). An introduction to latent semantic analysis. Discourse Processes, 25(2\u20133), 259\u2013284.","journal-title":"Discourse Processes"},{"key":"26_CR85","unstructured":"Leacock, C., & Chodorow, M. (1998). Combining local context and WordNet sense similarity for word sense identification. In C. Fellbaum (Ed.), WordNet: an electronic lexical database, language, speech, and communication (Chap. 11, pp. 265\u2013284). MIT Press."},{"issue":"4","key":"26_CR86","doi-asserted-by":"crossref","first-page":"389","DOI":"10.1023\/A:1025779619903","volume":"37","author":"C Leacock","year":"2003","unstructured":"Leacock, C., & Chodorow, M. (2003). C-rater: automated scoring of short-answer questions. Computers and the Humanities, 37(4), 389\u2013405.","journal-title":"Computers and the Humanities"},{"key":"26_CR87","unstructured":"Lesk, M. (1986). Automatic sense disambiguation using machine readable dictionaries: how to tell a pine cone from an ice cream cone. In V.D. Buys (Ed.), Proceedings of the 5th annual international conference on systems documentation (pp. 24\u201326). Toronto: ACM."},{"key":"26_CR88","unstructured":"Levy, O., Zesch, T., Dagan, I., Gurevych, I. (2013). Recognizing partial textual entailment. In H. Schuetze, P. Fung, M. Poesio (Eds.), Proceedings of the 51st annual meeting of the association for computational linguistics (Vol. 2, pp. 451\u2013455). Sofia: Association for Computational Linguistics."},{"key":"26_CR89","unstructured":"Lin, C.-Y. (2004). ROUGE: A package for automatic evaluation of summaries. In M.F. Moens & S. Szpakowicz (Eds.), Proceedings of the 1st text summarization branches out workshop at ACL (pp. 74\u201381). Barcelona: Association for Computational Linguistics."},{"key":"26_CR90","unstructured":"Lin, D. (1998). An information-theoretic definition of similarity. In J.W. Shavlik (Eds.), Proceedings of the 15th international conference on machine learning (pp. 296\u2013304). Madison: Morgan Kaufmann Publishers."},{"key":"26_CR91","unstructured":"Madnani, N., Burstein, J., Sabatini, J., Reilly, T.O. (2013). Automated scoring of a summary writing task designed to measure reading comprehension. In J. Tetreault, J. Burstein, C. Leacock (Eds.), Proceedings of the 8th workshop on innovative use of nlp for building educational applications (pp. 163\u2013168). Atlanta: Association for Computational Linguistics."},{"issue":"1","key":"26_CR92","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1080\/00031305.1975.10479105","volume":"29","author":"DW Marquardt","year":"1975","unstructured":"Marquardt, D.W., & Snee, R.D. (1975). Ridge regression in practice. The American Statistician, 29(1), 3\u201320.","journal-title":"The American Statistician"},{"issue":"2","key":"26_CR93","doi-asserted-by":"crossref","first-page":"151","DOI":"10.1207\/s15324818ame0502_5","volume":"5","author":"ME Martinez","year":"1992","unstructured":"Martinez, M.E., & Bennett, R.E. (1992). A review of automatically scorable constructed-response item types for large-scale assessment. Applied Measurement in Education, 5(2), 151\u2013169.","journal-title":"Applied Measurement in Education"},{"key":"26_CR94","unstructured":"Meurers, D., Ott, N., Ziai, R. (2010). Compiling a task-based corpus for the analysis of learner language in context. In O. Bott, S. Featherston, I. Steiner, B. Stolterfoht, Y. Versley (Eds.), Proceedings of the 4th linguistic evidence conference (pp. 214\u2013217). T\u00fcbingen."},{"key":"26_CR95","unstructured":"Meurers, D., Ziai, R., Ott, N., Kopp, J. (2011a). Evaluating answers to reading comprehension questions in context: results for german and the role of information structure. In P. Clark, I. Dagan, K. Erk, S. Pado, S. Thater, F.M. Zanzotto (Eds.), Proceedings of the 2nd textinfer workshop on textual entailment (pp. 1\u20139). Edinburgh: Association for Computational Linguistics."},{"issue":"4","key":"26_CR96","doi-asserted-by":"crossref","first-page":"355","DOI":"10.1504\/IJCEELL.2011.042793","volume":"21","author":"D Meurers","year":"2011b","unstructured":"Meurers, D., Ziai, R., Ott, N., Bailey, S.M. (2011b). Integrating parallel analysis modules to evaluate the meaning of answers to reading comprehension questions. International Journal of Continuing Engineering Education and Life-Long Learning, 21(4), 355\u2013369.","journal-title":"International Journal of Continuing Engineering Education and Life-Long Learning"},{"key":"26_CR97","unstructured":"Mitchell, T., Russell, T., Broomhead, P., Aldridge, N. (2002). Towards robust computerised marking of free-text responses. In Proceedings of the 6th computer assisted assessment conference (pp. 233\u2013249). Loughborough."},{"key":"26_CR98","unstructured":"Mitchell, T., Aldridge, N., Williamson, W., Broomhead, P. (2003a). Computer based testing of medical knowledge. In Proceedings of the 7th computer assisted assessment conference (pp. 249\u2013267). Loughborough."},{"key":"26_CR99","unstructured":"Mitchell, T., Aldridge, N., Broomhead, P. (2003b). Computerised marking of short-answer free-text responses. In Proceedings of the 29th annual conference of the international association for educational assessment (pp. 1\u201316). Manchester."},{"key":"26_CR100","unstructured":"Mohler, M., & Mihalcea, R. (2009). Text-to-text semantic similarity for automatic short answer grading. In A. Lascarides, C. Gardent, J. Nivre (Eds.), Proceedings of the 12th conference of the european chapter of the association for computational linguistics (pp. 567\u2013575). Athens: Association for Computational Linguistics."},{"key":"26_CR101","unstructured":"Mohler, M., Bunescu, R., Mihalcea, R. (2011). Learning to grade short answer questions using semantic similarity measures and dependency graph alignments. In D. Lin (Ed.), Proceedings of the 49th annual meeting of the association for computational linguistics: human language technologies volume 1 of HLT \u201911 (pp. 752\u2013762). Portland: Association for Computational Linguistics."},{"key":"26_CR102","unstructured":"Moser, J.R. (2009). The electronic assessor: design and prototype of an automated free-text assessment system. Master\u2019s Thesis, Institute for Information Systems and Computer Media, Technical University of Graz."},{"key":"26_CR103","unstructured":"Nielsen, R.D., Ward, W., Martin, J.H., Palmer, M. (2008a). Annotating students\u2019 understanding of science concepts. In N. Calzolari, K. Choukri, B. Maegaard, J. Mariani, J. Odijk, S. Piperidis, D. Tapias (Eds.), Proceedings of the 6th international conference on language resources and evaluation (pp. 1\u20138). Marrakech: European Language Resources Association."},{"key":"26_CR104","unstructured":"Nielsen, R.D., Ward, W., Martin, J.H. (2008b). Learning to assess low-level conceptual understanding. In D. Wilson & H.C. Lane (Eds.), Proceedings of the 21st international florida artificial intelligence research society conference (pp. 427\u2013432). Coconut Grove: AAAI Press."},{"issue":"4","key":"26_CR105","doi-asserted-by":"crossref","first-page":"479","DOI":"10.1017\/S135132490999012X","volume":"15","author":"RD Nielsen","year":"2009","unstructured":"Nielsen, R.D., Ward, W., Martin, J.H. (2009). Recognizing entailment in intelligent tutoring systems. Natural Language Engineering, 15(4), 479\u2013501.","journal-title":"Natural Language Engineering"},{"key":"26_CR106","unstructured":"Ott, N., Ziai, R., Meurers, D. (2012). Creation and analysis of a reading comprehension exercise corpus: towards evaluating meaning in context. In T. Schmidt & K. W\u00f6rner (Eds.), Multilingual corpora and multilingual corpus analysis volume 14 of hamburg studies on multilingualism (pp. 47\u201369). John Benjamins Publishing: Amsterdam."},{"key":"26_CR107","unstructured":"Ott, N., Ziai, R., Hahn, M., Meurers, D. (2013). CoMeT: integrating different levels of linguistic modeling for meaning assessment. In S. Manandhar & D. Yuret (Eds.), Proceedings of the 7th international workshop on semantic evaluation (pp. 608\u2013616). Atlanta: Association for Computational Linguistics."},{"issue":"5","key":"26_CR108","first-page":"238","volume":"47","author":"EB Page","year":"1966","unstructured":"Page, E.B. (1966). The imminence of grading essays by computer. Phi Delta Kappan, 47(5), 238\u2013243.","journal-title":"Phi Delta Kappan"},{"key":"26_CR109","unstructured":"Papadimitriou, C.H., Tamaki, H., Raghavan, P., Vempala, S. (1998). Latent semantic indexing: a probabilistic analysis. In A. Mendelson & J. Paredaens (Eds.), Proceedings of the 17th ACM SIGACT-SIGMOD-SIGART symposium on principles of database systems PODS \u201998 (pp. 159\u2013168). Seattle: ACM."},{"key":"26_CR110","unstructured":"Papineni, K., Roukos, S., Ward, T., Zhu, W.-J. (2002). BLEU: a method for automatic evaluation of machine translation. In P. Isabelle (Ed.), In Proceedings of the 40th annual meeting of the association for computational linguistics (pp. 311\u2013318). Philadelphia: Association for Computational Linguistics."},{"key":"26_CR111","unstructured":"Pascual-Nieto, I., Perez-Marin, D., O\u2019Donnell, M., Rodriguez, P. (2008). Enhancing a free-text adaptive computer assisted assessment system with self-assessment features. In P. D\u00edaz, I.A. Kinshuk, E. Mora (Eds.), Proceedings of the 8th IEEE international conference on advanced learning technologies (pp. 399\u2013401). Santander: IEEE."},{"key":"26_CR112","unstructured":"Pascual-Nieto, I., Santos, O.C., Perez-Marin, D., Boticario, J.G. (2011). Extending computer assisted assessment systems with natural language processing, user modeling and recommendations based on human computer interaction and data mining. In T. Walsh (Ed.), Proceedings of the 22nd international joint conference on artificial intelligence, volume 3 of IJCAI \u201911 (pp. 2519\u20132524). Barcelona: AAAI Press."},{"key":"26_CR113","unstructured":"Pearson Education (2010). Intelligent Essay Assessor (IEA) Fact Sheet. Online Brochure. http:\/\/kt.pearsonassessments.com\/download\/IEA-FactSheet-20100401.pdf ."},{"key":"26_CR114","unstructured":"Pedersen, T., Patwardhan, S., Michelizzi, J. (2004). WordNet::Similarity: measuring the relatedness of concepts. In D. Palmer, J. Polifroni, D. Roy (Eds.), Proceedings of the human language technology conference of the north american chapter of the association for computational linguistics (Demonstration Papers) (pp. 38\u201341). Boston: Association for Computational Linguistics."},{"key":"26_CR115","unstructured":"P\u00e9rez, D., & Alfonseca, E. (2005). Adapting the automatic assessment of free-text answers to the students. In Proceedings of the 9th computer assisted assessment conference (pp. 1\u201312). Loughborough."},{"key":"26_CR116","unstructured":"P\u00e9rez, D., Alfonseca, E., Rodr\u00edguez, P. (2004). Application of the BLEU method for evaluating free-text answers in an e-learning environment. In M.T. Lino, M.F. Xavier, F. Ferreira, R. Costa, R. Silva (Eds.), Proceedings of the 4th international conference on language resources and evaluation (pp. 1351\u20131354). Lisbon."},{"key":"26_CR117","unstructured":"P\u00e9rez, D., Alfonseca, E., Rodr\u00edguez, P. (2004). Upper bounds of the BLEU algorithm applied to assessing student essays. In Proceedings of the 30th international association for educational assessment conference. Philadelphia."},{"issue":"59","key":"26_CR118","first-page":"325","volume":"38","author":"D P\u00e9rez","year":"2005","unstructured":"P\u00e9rez, D., Alfonseca, E., Rodr\u00edguez, P., Gliozzo, A., Strapparava, C., Magnini, B. (2005a). About the effects of combining latent semantic analysis with natural language processing techniques for free-text assessment. Revista Signos: Estudios de Ling\u00fc\u00edstica, 38(59), 325\u2013343.","journal-title":"Revista Signos: Estudios de Ling\u00fc\u00edstica"},{"key":"26_CR119","unstructured":"P\u00e9rez, D., Postolache, O., Alfonseca, E., Cristea, D., Rodr\u00edguez, P. (2005b). About the effects of using anaphora resolution in assessing free-text student answers. In R. Mitkov (Ed.), Proceedings of the 11th international conference on recent advances in natural language processing (pp. 380\u2013386). Borovets."},{"key":"26_CR120","unstructured":"P\u00e9rez, D., Gliozzo, A.M., Strapparava, C., Alfonseca, E., Rodr\u00edguez, P., Magnini, B. (2005c). Automatic assessment of students\u2019 free-text answers underpinned by the combination of a BLEU-inspired algorithm and latent semantic analysis. In D. Cook, L. Holder, I. Russell, Z. Markov (Eds.), Proceedings of the 18th international florida artificial intelligence research society conference (pp. 358\u2013363). Clearwater Beach: AAAI Press."},{"key":"26_CR121","unstructured":"P\u00e9rez-Mar\u00edn, D. (2004). Automatic evaluation of user\u2019s short essays by using statistical and shallow natural language processing techniques. Diploma thesis, Computer Science Department, Universidad Aut\u00f3noma of Madrid."},{"key":"26_CR122","unstructured":"P\u00e9rez-Mar\u00edn, D. (2007). Adaptive computer assisted assessment of free-text students\u2019 answers: an approach to automatically generate students\u2019 conceptual models. Ph.D. thesis, Computer Science Department, Universidad Aut\u00f3noma of Madrid."},{"issue":"2","key":"26_CR123","doi-asserted-by":"crossref","first-page":"155","DOI":"10.1504\/IJCEELL.2011.040196","volume":"21","author":"D P\u00e9rez-Mar\u00edn","year":"2011","unstructured":"P\u00e9rez-Mar\u00edn, D., & Pascual-Nieto, I. (2011). Willow: a system to automatically assess students\u2019 free-text answers by using a combination of shallow nlp techniques. International Journal of Continuing Engineering Education and Life Long Learning, 21(2), 155\u2013169.","journal-title":"International Journal of Continuing Engineering Education and Life Long Learning"},{"key":"26_CR124","unstructured":"P\u00e9rez-Mar\u00edn, D., Alfonseca, E., Rodr\u00edguez, P. (2006a). A free-text scoring system that generates conceptual models of the students\u2019 knowledge with the aid of clarifying questions. In L. Aroyo & D. Dicheva (Eds.), Proceedings of the 4th international workshop on applications of semantic web technologies for e-learning (pp. 1\u20132). Dublin."},{"key":"26_CR125","doi-asserted-by":"crossref","unstructured":"P\u00e9rez-Mar\u00edn, D., Alfonseca, E., Freire, M., Rodr\u00edguez, P., Guirao, J.M., Moreno-Sandoval, A. (2006b). Automatic generation of students\u2019 conceptual models underpinned by free-text adaptive computer assisted assessment. In R. Kinshuk, P. Koper, P. Kommers, D.S. Kirschner, W. Didderen (Eds.), Proceedings of the 6th international conference on advanced learning technologies (pp. 280\u2013284). Kerkrade: IEEE.","DOI":"10.1109\/ICALT.2006.1652424"},{"key":"26_CR126","doi-asserted-by":"crossref","unstructured":"P\u00e9rez-Mar\u00edn, D., Alfonseca, E., Rodr\u00edguez, P. (2006c). On the dynamic adaptation of computer assisted assessment of free-text answers. In V.P. Wade, H. Ashman, B. Smyth (Eds.), Proceedings of the 4th international conference on adaptive hypermedia and adaptive web-based systems volume 4018 of lecture notes in computer science (pp. 374\u2013377). Dublin.","DOI":"10.1007\/11768012_54"},{"key":"26_CR127","unstructured":"P\u00e9rez-Mar\u00edn, D., Alfonseca, E., Rodr\u00edguez, P., Pascual-Nieto, I. (2006d). Willow: automatic and adaptive assessment of students\u2019 free-text answers. In F. Pla (Ed.), Proceedings of the 22nd international conference of the spanish society for natural language processing (pp. 367\u2013368). Zaragoza."},{"key":"26_CR128","unstructured":"P\u00e9rez-Mar\u00edn, D., Pascual-Nieto, I., Alfonseca, E., Anguiano, E. (2007). A study on the impact of the use of an automatic and adaptive free-text assessment system during a university course. In J. Fong & F.L. Wang (Eds.), Proceedings of the workshop on blended learning ICWL \u201907 (pp. 186\u2013195). Edinburgh: Pearson."},{"issue":"4","key":"26_CR129","doi-asserted-by":"crossref","first-page":"353","DOI":"10.1017\/S026988890999018X","volume":"24","author":"D P\u00e9rez-Mar\u00edn","year":"2009","unstructured":"P\u00e9rez-Mar\u00edn, D., Pascual-Nieto, I., Rodr\u00edguez, P. (2009). Computer-assisted assessment of free-text answers. The Knowledge Engineering Review, 24(4), 353\u2013374.","journal-title":"The Knowledge Engineering Review"},{"key":"26_CR130","unstructured":"Potthast, M. (2011). Technologies for reusing text from the Web. Ph.D. thesis. Weimar: Bauhaus-Universit\u00e4t Weimar."},{"key":"26_CR131","first-page":"13:1","volume":"3","author":"P Prettenhofer","year":"2011","unstructured":"Prettenhofer, P., & Stein, B. (2011). Cross-lingual adaptation using structural correspondence learning. Transactions on Intelligent Systems and Technology, 3, 13:1\u201313:22.","journal-title":"Transactions on Intelligent Systems and Technology"},{"key":"26_CR132","unstructured":"Pulman, S.G., & Sukkarieh, J.Z. (2005). Automatic short answer marking. In J. Burstein & C. Leacock (Eds.), Proceedings of the 2nd workshop on building educational applications using NLP (pp. 9\u201316). Ann Arbor: Association for Computational Linguistics."},{"key":"26_CR133","unstructured":"Resnik, P. (1995). Using information content to evaluate semantic similarity in a taxonomy. In C.R. Perrault & C.S. Mellish (Eds.), Proceedings of the 14th international joint conference on artificial intelligence volume 1 of IJCAI \u201995 (pp. 448\u2013453). Montreal: Morgan Kaufmann Publishers."},{"key":"26_CR134","unstructured":"Richter, F., & Sailer, M. (2003). Basic concepts of lexical resource semantics. In A. Beckmann & N. Preining (Eds.), Proceedings of the 15th european summer school in logic language and information volume 5 of collegium logicum (pp. 87\u2013143). Vienna: Kurt G\u00f6del Society."},{"issue":"1","key":"26_CR135","doi-asserted-by":"crossref","first-page":"59","DOI":"10.2307\/2685263","volume":"42","author":"JL Rodgers","year":"1988","unstructured":"Rodgers, J.L., & Nicewander, W.A. (1988). Thirteen ways to look at the correlation coefficient. The American Statistician, 42(1), 59\u201366.","journal-title":"The American Statistician"},{"key":"26_CR136","unstructured":"Sargeant, J., McGee Wood, M., Anderson, S.M. (2004). A human-computer collaborative approach to the marking of free text answers. In Proceedings of the 8th computer assisted assessment conference (pp. 361\u2013370). Loughborough: Loughborough University."},{"key":"26_CR137","doi-asserted-by":"crossref","unstructured":"Shermis, M.D., & Burstein, J. (2003). Automated essay scoring: a cross-disciplinary perspective, 1st edn. Mahwah: Lawrence Erlbaum Associates.","DOI":"10.4324\/9781410606860"},{"key":"26_CR138","doi-asserted-by":"crossref","unstructured":"Shermis, M.D., & Burstein, J. (2013). Handbook of automated essay evaluation: current applications and new directions, 1st edn. New York: Routledge New York City.","DOI":"10.4324\/9780203122761"},{"key":"26_CR139","unstructured":"Shermis, M.D., Burstein, J., Leacock, C. (2008). Applications of computers in assessment and analysis of writing. In C.A. MacArthur, S. Graham, J. Fitzgerald (Eds.), Handbook of writing research, chapter 27, 1st edn. (pp. 403\u2013416). New York: Guilford Press New York City."},{"key":"26_CR140","doi-asserted-by":"crossref","unstructured":"Siddiqi, R., & Harrison, C.J. (2008a). A systematic approach to the automated marking of short-answer questions. In M.K. Anis, M.K. Khan, S.J.H. Zaidi (Eds.), Proceedings of the 12th international multitopic conference (pp. 329\u2013332). Karachi: IEEE.","DOI":"10.1109\/INMIC.2008.4777758"},{"key":"26_CR141","unstructured":"Siddiqi, R., & Harrison, C.J. (2008b). On the automated assessment of short free-text responses. In Proceedings of the 34th international association for educational assessment annual conference (pp. 1\u201311). Cambridge."},{"issue":"3","key":"26_CR142","doi-asserted-by":"crossref","first-page":"237","DOI":"10.1109\/TLT.2010.4","volume":"3","author":"R Siddiqi","year":"2010","unstructured":"Siddiqi, R., Harrison, C.J., Siddiqi, R. (2010). Improving teaching and learning through automated short-answer marking. IEEE Transactions on Learning Technologies, 3(3), 237\u2013249.","journal-title":"IEEE Transactions on Learning Technologies"},{"issue":"3","key":"26_CR143","doi-asserted-by":"crossref","first-page":"257","DOI":"10.1093\/ptj\/85.3.257","volume":"85","author":"J Sim","year":"2005","unstructured":"Sim, J., & Wright, C.C. (2005). The kappa statistic in reliability studies: use, interpretation, and sample size requirements. Physical Therapy, 85(3), 257\u2013268.","journal-title":"Physical Therapy"},{"key":"26_CR144","unstructured":"Sima, D., Schmuck, B., Sz\u00f6ll, S., Mikl\u00f3s, A. (2007). Intelligent short text assessment in eMax. In Proceedings of the 8th Africon conference (pp. 1\u20136). Windhoek: IEEE."},{"key":"26_CR145","doi-asserted-by":"crossref","unstructured":"Sima, D., Schmuck, B., Sz\u00f6ll, S., Mikl\u00f3s, A. (2009). Intelligent short text assessment in eMax. In I.J. Rudas, J. Fodor, J. Kacprzyk (Eds.), Towards intelligent engineering and information technology volume 243 of studies in computational intelligence (pp. 435\u2013445). Springer.","DOI":"10.1007\/978-3-642-03737-5_31"},{"issue":"1","key":"26_CR146","doi-asserted-by":"crossref","first-page":"72","DOI":"10.2307\/1412159","volume":"15","author":"C Spearman","year":"1904","unstructured":"Spearman, C. (1904). The proof and measurement of association between two things. The American Journal of Psychology, 15(1), 72\u2013101.","journal-title":"The American Journal of Psychology"},{"key":"26_CR147","unstructured":"Stern, A., & Dagan, I. (2011). A confidence model for syntactically-motivated entailment proofs. In R. Mitkov & G. Angelova (Eds.), Proceedings of the 14th international conference on recent advances in natural language processing (pp. 455\u2013462). Hissar: Association for Computational Linguistics."},{"issue":"2684","key":"26_CR148","doi-asserted-by":"crossref","first-page":"667","DOI":"10.1126\/science.103.2684.677","volume":"103","author":"SS Stevens","year":"1946","unstructured":"Stevens, S.S. (1946). On the theory of scales of measurement. Science, 103(2684), 667\u2013680.","journal-title":"Science"},{"key":"26_CR149","unstructured":"Sukkarieh, J.Z. (2010). Using a MaxEnt classifier for the automatic content scoring of free-text responses. In A. Mohammad-Djafari, J.-F. Bercher, P. Bessi\u00e9re (Eds.), Proceedings of the international workshop on bayesian inference and maximum entropy methods in science and engineering volume 1305 of aip conference proceedings (pp. 41\u201318). Chamonix: American Institute of Physics."},{"key":"26_CR150","unstructured":"Sukkarieh, J.Z., & Blackmore, J. (2009). c-rater: automatic content scoring for short constructed responses. In H.C. Lane & H.W. Guesgen (Eds.), Proceedings of the 22nd international conference of the florida artificial intelligence research society (pp. 290\u2013295). Sanibel Island: AAAI Press."},{"key":"26_CR151","unstructured":"Sukkarieh, J.Z., & Bolge, E. (2008). Leveraging c-rater\u2019s automated scoring capability for providing instructional feedback for short constructed responses. In B.P. Woolf, E. A\u00efmeur, R. Nkambou, S. Lajoie (Eds.), Proceedings of the 9th international conference on intelligent tutoring systems ITS \u201908 (pp. 779\u2013783). Montreal: Springer."},{"key":"26_CR152","unstructured":"Sukkarieh, J.Z., & Bolge, E. (2010). Building a textual entailment suite for evaluating content scoring technologies. In N. Calzolari, K. Choukri, B. Maegaard, J. Mariani, J. Odijk, S. Piperidis, M. Rosner, D. Tapias (Eds.), Proceedings of the 5th international conference on language resources and evaluation, LREC \u201910 (pp. 1\u20138). Valletta: European Language Resources Association."},{"key":"26_CR153","unstructured":"Sukkarieh, J.Z., & Kamal, J. (2009). Towards Agile and Test-Driven Development in NLP Applications. In K.B. Cohen & M. Light (Eds.), Proceedings of the workshop on software engineering, testing, and quality assurance for natural language processing, SETQA-NLP \u201909 (pp. 42\u201344). Boulder: Association for Computational Linguistics."},{"key":"26_CR154","unstructured":"Sukkarieh, J.Z., & Pulman, S.G. (2005). Information extraction and machine learning: auto-marking short free text responses to science questions. In C.-K. Looi, G.I. McCalla, B. Bredeweg, J. Breuker (Eds.), Proceedings of the 12th international conference on artificial intelligence in education, frontiers in artificial intelligence and applications (pp. 629\u2013637). Amsterdam: IOS Press."},{"key":"26_CR155","unstructured":"Sukkarieh, J.Z., & Stoyanchev, S. (2009). Automating model building in c-rater. In C. Callison-Burch & F.M. Zanzotto (Eds.), Proceedings of the 1st ACL\/IJCNLP workshop on applied textual inference, TextInfer \u201909 (pp. 61\u201369). Suntec: Association for Computational Linguistics."},{"key":"26_CR156","unstructured":"Sukkarieh, J.Z., Pulman, S.G., Raikes, N. (2003). Auto-marking: using computational linguistics to score short, free text responses. In Proceedings of the 29th annual conference of the international association for educational assessment (pp. 1\u201315). Manchester."},{"key":"26_CR157","unstructured":"Sukkarieh, J.Z., Pulman, S.G., Raikes, N. (2004). Auto-marking 2: an update on the ucles-oxford university research into using computational linguistics to score short, free text responses. In Proceedings of the 30th annual conference of the international association for educational assessment. Philadelphia."},{"key":"26_CR158","unstructured":"Swithenby, S., & Jordan, S. (2008). Supporting open learners by computer based assessment with short free-text responses and tailored feedback. In F. Welsch, F. Malpica, A. Tremante, J.V. Carrasquero, A. Oropeza (Eds.), Proceedings of the 2nd international multi-conference on society, cybernetics and informatics IMSCI \u201908. Orlando: International Institute of Informatics and Systemics."},{"key":"26_CR159","unstructured":"Szpektor, I., & Dagan, I. (2007). Learning canonical forms of entailment rules. In R. Mitkov & G. Angelova (Eds.), Proceedings of the 6th international conference on recent advances in natural language processing (pp. 1\u20136). Borovets."},{"key":"26_CR160","unstructured":"Tandalla, L. (2012). Scoring short answer essays. ASAP \u201912 SAS Methodology Paper."},{"key":"26_CR161","unstructured":"Thomas, P. (2003). The evaluation of electronic marking of examinations. In Proceedings of the 8th annual conference on innovation and technology in computer science education, ITiCSE \u201903 (pp. 50\u201354). Thessaloniki: ACM."},{"key":"26_CR162","doi-asserted-by":"crossref","first-page":"319","DOI":"10.28945\/331","volume":"2","author":"S Valenti","year":"2003","unstructured":"Valenti, S., Neri, F., Cucchiarelli, A. (2003). An overview of current research on automated essay grading. Journal of Information Technology Education, 2, 319\u2013330.","journal-title":"Journal of Information Technology Education"},{"key":"26_CR163","unstructured":"VanLehn, K., Jordan, P.W., Ros\u00e9, C.P., Bhembe, D., B\u00f6ttner, M., Gaydos, A., Makatchev, M., Pappuswamy, U., Ringenberg, M., Roque, A., Siler, S., Srivastava, R. (2002). The architecture of why2-atlas: a coach for qualitative physics essay writing. In S.A. Cerri, G. Gouarderes, F. Paraguacu (Eds.), Proceedings of the sixth international conference on intelligent tutoring systems volume 2363 of lecture notes in computer science (pp. 158\u2013167). Biarritz: Springer."},{"key":"26_CR164","unstructured":"Wachsmuth, H., Stein, B., Engels, G. (2011). Constructing efficient information extraction pipelines. In B. Berendt, A. de Vries, W. Fan, C. MacDonald, I. Ounis, I. Ruthven (Eds.), Proceedings of the 12th ACM international conference on information and knowledge management, CIKM \u201911 (pp. 2237\u20132240). Glasgow: ACM."},{"key":"26_CR165","unstructured":"Wachsmuth, H., Stein, B., Engels, G. (2013). Information extraction as a filtering task. In Q. He & A. Iyengar (Eds.), In Proceedings of the 22nd ACM international conference on information and knowledge management CIKM \u201913 (pp. 2049\u20132058). San Francisco: ACM."},{"issue":"4","key":"26_CR166","doi-asserted-by":"crossref","first-page":"1450","DOI":"10.1016\/j.compedu.2008.01.006","volume":"51","author":"H-C Wang","year":"2008","unstructured":"Wang, H.-C., Chang, C.-Y., Li, T.-Y. (2008). Assessing creative problem-solving with automated text grading. Computers & Education, 51(4), 1450\u20131466.","journal-title":"Computers & Education"},{"key":"26_CR167","unstructured":"Wang, X., Evanini, K., Zechner, K. (2013). Coherence modeling for the automated assessment of spontaneous spoken responses. In L. Vanderwende, H. Daum\u00e9, K. Kirchhoff (Eds.), Proceedings of the conference of the North American chapter of the association for computational linguistics: human language technologies (pp. 81\u2013819). Atlanta: Association for Computational Linguistics."},{"issue":"1","key":"26_CR168","doi-asserted-by":"crossref","first-page":"2","DOI":"10.1111\/j.1745-3992.2011.00223.x","volume":"31","author":"DM Williamson","year":"2012","unstructured":"Williamson, D.M., Xi, X., Breyer, F.J. (2012). A framework for evaluation and use of automated scoring. Educational Measurement: Issues and Practice, 31(1), 2\u201313.","journal-title":"Educational Measurement: Issues and Practice"},{"key":"26_CR169","unstructured":"Willis, A. (2010). Inductive logic programming to support automatic marking of student answers in free text. Final report, Milton Keynes: COLMSCT, The Open University."},{"key":"26_CR170","unstructured":"Wise, M.J. (1993). String similarity via greedy string tiling and running Karp-Rabin matching. Technical report, Department of Computer Science, University of Sydney."},{"key":"26_CR171","unstructured":"Wood, M.M., Jones, C., Sargeant, J., Reed, P. (2006). Light-weight clustering techniques for short text answers in HCC CAA. In M. Danson (Ed.), Proceedings of the 10th CAA international computer assisted assessment conference (pp. 291\u2013308). Loughborough: Loughborough University."},{"key":"26_CR172","unstructured":"Wu, Z., & Palmer, M. (1994). Verb semantics and lexical selection. In P. Justejovsky (Ed.) Proceedings of the 32nd annual meeting on association for computational linguistics ACL \u201994 (pp. 133\u2013138). Las Cruces: Association for Computational Linguistics."},{"key":"26_CR173","unstructured":"Zbontar, J. (2012). Short answer scoring by stacking. ASAP \u201912 SAS methodology paper."},{"issue":"4","key":"26_CR174","doi-asserted-by":"crossref","first-page":"337","DOI":"10.1207\/S15324818AME1504_02","volume":"15","author":"AL Zenisky","year":"2002","unstructured":"Zenisky, A.L., & Sireci, S.G. (2002). Technological innovations in large-scale assessment. Applied Measurement in Education, 15(4), 337\u2013362.","journal-title":"Applied Measurement in Education"},{"key":"26_CR175","unstructured":"Zesch, T., Levy, O., Gurevych, I., Dagan, I. (2013). UKP-BIU: similarity and entailment metrics for student response analysis. In S. Manandhar & D. Yuret (Eds.), Proceedings of the 17th international workshop on semantic evaluation (Vol. 2, pp. 285\u2013289). Atlanta: Association for Computational Linguistics."},{"key":"26_CR176","unstructured":"Ziai, R., Ott, N., Meurers, D. (2012). Short answer assessment: establishing links between research strands. In J. Tetreault, J. Burstein, C. Leacock (Eds.), Proceedings of the 17th workshop on the innovative use of NLP for building educational applications (pp. 190\u2013200). Montreal: Association for Computational Linguistics."}],"container-title":["International Journal of Artificial Intelligence in Education"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/link.springer.com\/content\/pdf\/10.1007\/s40593-014-0026-8.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"http:\/\/link.springer.com\/article\/10.1007\/s40593-014-0026-8\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"http:\/\/link.springer.com\/content\/pdf\/10.1007\/s40593-014-0026-8","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,3,4]],"date-time":"2026-03-04T18:11:48Z","timestamp":1772647908000},"score":1,"resource":{"primary":{"URL":"http:\/\/link.springer.com\/10.1007\/s40593-014-0026-8"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2014,10,23]]},"references-count":176,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2015,3]]}},"alternative-id":["26"],"URL":"https:\/\/doi.org\/10.1007\/s40593-014-0026-8","relation":{},"ISSN":["1560-4292","1560-4306"],"issn-type":[{"value":"1560-4292","type":"print"},{"value":"1560-4306","type":"electronic"}],"subject":[],"published":{"date-parts":[[2014,10,23]]}}}