{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,12,5]],"date-time":"2025-12-05T23:30:32Z","timestamp":1764977432577,"version":"3.46.0"},"reference-count":41,"publisher":"Walter de Gruyter GmbH","issue":"2","license":[{"start":{"date-parts":[[2016,5,11]],"date-time":"2016-05-11T00:00:00Z","timestamp":1462924800000},"content-version":"unspecified","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by-nc-nd\/3.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2017,4,1]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>The assessment of answers is an important process that requires great effort from evaluators. This assessment process requires high concentration without any fluctuations in mood. This substantiates the need to automate answer script evaluation. Regarding text answer evaluation, sentence similarity measures have been widely used to compare student written answers with reference texts. In this paper, we propose an automated answer evaluation system that uses our proposed cosine-based sentence similarity measures to evaluate the answers. Cosine measures have proved to be effective in comparing between free text student answers and reference texts. Here we propose a set of novel cosine-based sentence similarity measures with varied approaches of creating document vector space. In addition to this, we propose a novel synset-based word similarity measure for computation of document vectors coupled with varied approaches for dimensionality-reduction for reducing vector space dimensions. Thus, we propose 21 cosine-based sentence similarity measures and measured their performance using MSR paraphrase corpus and Li\u2019s benchmark datasets. We also use these measures for automatic answer evaluation system and compare their performances using the Kaggle short answer and essay dataset. The performance of the system-generated scores is compared with the human scores using Pearson correlation. The results show that system and human scores have correlation between each other.<\/jats:p>","DOI":"10.1515\/jisys-2015-0031","type":"journal-article","created":{"date-parts":[[2016,5,11]],"date-time":"2016-05-11T06:02:43Z","timestamp":1462946563000},"page":"243-262","source":"Crossref","is-referenced-by-count":1,"title":["Design and Development of a Framework for an Automatic Answer Evaluation System Based on Similarity Measures"],"prefix":"10.1515","volume":"26","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-6765-5624","authenticated-orcid":false,"given":"Madhumitha","family":"Ramamurthy","sequence":"first","affiliation":[{"name":"Department of Computer Science and Engineering, Sri Krishna College of Engineering and Technology, Coimbatore 641008, Tamil Nadu, India"}]},{"given":"Ilango","family":"Krishnamurthi","sequence":"additional","affiliation":[{"name":"Department of Computer Science and Engineering, Sri Krishna College of Engineering and Technology, Coimbatore 641008, Tamil Nadu, India"}]}],"member":"374","published-online":{"date-parts":[[2016,5,11]]},"reference":[{"key":"2025120523263527558_j_jisys-2015-0031_ref_001_w2aab3b7c79b1b6b1ab2ab1Aa","doi-asserted-by":"crossref","unstructured":"J. A. Aslam and M. Frost, An information-theoretic measure for document similarity, in: Proceedings of the 26th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 449\u2013450, ACM, New York, NY, USA, 2003.","DOI":"10.1145\/860435.860545"},{"key":"2025120523263527558_j_jisys-2015-0031_ref_002_w2aab3b7c79b1b6b1ab2ab2Aa","doi-asserted-by":"crossref","unstructured":"E. Bingham, and H. Mannila, Random projection in dimensionality reduction: applications to image and text data, in: Proceedings of the Seventh ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 245\u2013250, ACM, 2001.","DOI":"10.1145\/502512.502546"},{"key":"2025120523263527558_j_jisys-2015-0031_ref_003_w2aab3b7c79b1b6b1ab2ab3Aa","unstructured":"J. Burstein, K. Kukich, S. Wolff, L. Chi and M. Chodorow, Enriching automated essay scoring using discourse marking, in: Proceedings of the Workshop on Discourse Relations and Discourse Marking, Annual Meeting of the Association of Computational Linguistics, Montreal, Canada, 1998."},{"key":"2025120523263527558_j_jisys-2015-0031_ref_004_w2aab3b7c79b1b6b1ab2ab4Aa","unstructured":"J. Burstein, C. Leacock and R. Swartz, Automated evaluation of essays and short answers, in: Proceedings of the 6th International Computer Assisted Conference, edited by M. Danson, Loughborough, UK, 2001."},{"key":"2025120523263527558_j_jisys-2015-0031_ref_005_w2aab3b7c79b1b6b1ab2ab5Aa","doi-asserted-by":"crossref","unstructured":"H. Chim and X. Deng, Efficient phrase-based document similarity for clustering, IEEE Trans. Knowledge Data Eng.20 (2008), 1217\u20131229.","DOI":"10.1109\/TKDE.2008.50"},{"key":"2025120523263527558_j_jisys-2015-0031_ref_006_w2aab3b7c79b1b6b1ab2ab6Aa","doi-asserted-by":"crossref","unstructured":"C. G. Gonz\u00e1lez, W. Bonventi Jr. and A. V. Rodrigues, Density of closed balls in real-valued and autometrized boolean spaces for clustering applications, in: Advances in Artificial Intelligence-SBIA 2008, pp. 8\u201322, Springer, Berlin, Heidelberg, 2008.","DOI":"10.1007\/978-3-540-88190-2_7"},{"key":"2025120523263527558_j_jisys-2015-0031_ref_007_w2aab3b7c79b1b6b1ab2ab7Aa","doi-asserted-by":"crossref","unstructured":"R. W. Hamming, Error detecting and error correcting codes, Bell Syst. Tech. J.2b9 (1950), 147\u2013160.","DOI":"10.1002\/j.1538-7305.1950.tb00463.x"},{"key":"2025120523263527558_j_jisys-2015-0031_ref_008_w2aab3b7c79b1b6b1ab2ab8Aa","unstructured":"J. Han, M. Kamber and J. Pei, Data mining: concepts and techniques, Morgan Kaufmann Publishers, Burlington, MA, USA, 2011."},{"key":"2025120523263527558_j_jisys-2015-0031_ref_009_w2aab3b7c79b1b6b1ab2ab9Aa","doi-asserted-by":"crossref","unstructured":"M. A. Hearst, The debate on automated essay grading, IEEE Intell. Syst.15 (2000), 22\u201337.","DOI":"10.1109\/5254.889104"},{"key":"2025120523263527558_j_jisys-2015-0031_ref_010_w2aab3b7c79b1b6b1ab2ac10Aa","unstructured":"C. Ho, M. A. A. Murad, R. A. Kadir and S. C. Doraisamy, Word sense disambiguation-based sentence similarity, in: Proceedings of the 23rd International Conference on Computational Linguistics: Posters, pp. 418\u2013426, Association for Computational Linguistics, Stroudsburg, PA, USA, 2010."},{"key":"2025120523263527558_j_jisys-2015-0031_ref_011_w2aab3b7c79b1b6b1ab2ac11Aa","doi-asserted-by":"crossref","unstructured":"X. Hu, and H. Xia, Automated assessment system for subjective questions based on LSI, in: Intelligent Information Technology and Security Informatics (IITSI), 2010 Third International Symposium on, pp. 250\u2013254, IEEE, 2010.","DOI":"10.1109\/IITSI.2010.76"},{"key":"2025120523263527558_j_jisys-2015-0031_ref_012_w2aab3b7c79b1b6b1ab2ac12Aa","unstructured":"J. Jerrams-Smith, V. Soh and D. Callear Bridging gaps in computerized assessment of texts, in: Proceedings of the International Conference on Advanced Learning Technologies, pp. 139\u2013140, IEEE, 2001."},{"key":"2025120523263527558_j_jisys-2015-0031_ref_013_w2aab3b7c79b1b6b1ab2ac13Aa","unstructured":"T. Joachims, A Probabilistic Analysis of the Rocchio Algorithm with TFIDF for Text Categorization, No. CMU-CS-96-118, Department of Computer Science, Carnegie-Mellon University, Pittsburgh, PA, 1996."},{"key":"2025120523263527558_j_jisys-2015-0031_ref_014_w2aab3b7c79b1b6b1ab2ac14Aa","doi-asserted-by":"crossref","unstructured":"T. Joachims and F. Sebastiani, Guest editors\u2019 introduction to the special issue on automated text categorization, J. Intell. Inf. Syst.18 (2002), 103\u2013105.","DOI":"10.1023\/A:1013652626023"},{"key":"2025120523263527558_j_jisys-2015-0031_ref_015_w2aab3b7c79b1b6b1ab2ac15Aa","doi-asserted-by":"crossref","unstructured":"D. Kanejiya, A. Kumar, and S. Prasad, Automatic evaluation of students\u2019 answers using syntactically enhanced LSA, in: Proceedings of the HLT-NAACL 03 Workshop on Building Educational Applications Using Natural Language Processing, vol. 2, pp. 53\u201360, Association for Computational Linguistics, 2003.","DOI":"10.3115\/1118894.1118902"},{"key":"2025120523263527558_j_jisys-2015-0031_ref_016_w2aab3b7c79b1b6b1ab2ac16Aa","unstructured":"H. Kim, P. Howland and H. Park, Dimension reduction in text classification with support vector machines, J. Machine Learn. Res.6 (2005), 37\u201353."},{"key":"2025120523263527558_j_jisys-2015-0031_ref_017_w2aab3b7c79b1b6b1ab2ac17Aa","doi-asserted-by":"crossref","unstructured":"K. Knight, Mining online text, Commun. ACM42 (1999), 58\u201361.","DOI":"10.1145\/319382.319394"},{"key":"2025120523263527558_j_jisys-2015-0031_ref_018_w2aab3b7c79b1b6b1ab2ac18Aa","doi-asserted-by":"crossref","unstructured":"S. Kullback and R. A. Leibler, On information and sufficiency, Ann. Math. Stat.22 (1951), 79\u201386.","DOI":"10.1214\/aoms\/1177729694"},{"key":"2025120523263527558_j_jisys-2015-0031_ref_019_w2aab3b7c79b1b6b1ab2ac19Aa","doi-asserted-by":"crossref","unstructured":"A. B. Lajis, and N. A. Aziz, Part-of-speech in a node-link scoring techniques for assessing learners\u2019 understanding, Procedia Soc. Behav. Sci.27 (2011), 131\u2013139.","DOI":"10.1016\/j.sbspro.2011.10.591"},{"key":"2025120523263527558_j_jisys-2015-0031_ref_020_w2aab3b7c79b1b6b1ab2ac20Aa","doi-asserted-by":"crossref","unstructured":"M. C. Lee, A novel sentence similarity measure for semantic-based expert systems, Expert Syst. Appl.38 (2011), 6392\u20136399.","DOI":"10.1016\/j.eswa.2010.10.043"},{"key":"2025120523263527558_j_jisys-2015-0031_ref_021_w2aab3b7c79b1b6b1ab2ac21Aa","doi-asserted-by":"crossref","unstructured":"M. C. Lee, J. W. Chang and T. C. Hsieh, A grammar-based semantic similarity algorithm for natural language sentences, Sci. World J.2014 (2014), 437162.","DOI":"10.1155\/2014\/437162"},{"key":"2025120523263527558_j_jisys-2015-0031_ref_022_w2aab3b7c79b1b6b1ab2ac22Aa","doi-asserted-by":"crossref","unstructured":"Y. Li, D. McLean, Z. A. Bandar, J. D. O\u2019Shea, and K. Crockett, Sentence similarity based on semantic nets and corpus statistics, Knowledge and Data Engineering, IEEE Transactions on18 (2006), 1138\u20131150.","DOI":"10.1109\/TKDE.2006.130"},{"key":"2025120523263527558_j_jisys-2015-0031_ref_023_w2aab3b7c79b1b6b1ab2ac23Aa","doi-asserted-by":"crossref","unstructured":"L. Li, X. Hu, X. Hu, J. Wang, and Y. M. Zhou, Measuring sentence similarity from different aspects, in: Machine Learning and Cybernetics, 2009 International Conference on, vol. 4, pp. 2244\u20132249, IEEE, 2009.","DOI":"10.1109\/ICMLC.2009.5212182"},{"key":"2025120523263527558_j_jisys-2015-0031_ref_024_w2aab3b7c79b1b6b1ab2ac24Aa","unstructured":"D. Lin, An information-theoretic definition of similarity, ICML98 (1998), 296\u2013304."},{"key":"2025120523263527558_j_jisys-2015-0031_ref_025_w2aab3b7c79b1b6b1ab2ac25Aa","doi-asserted-by":"crossref","unstructured":"M. G. Michie, Use of the Bray-Curtis similarity measure in cluster analysis of foraminiferal data, J. Int. Assoc. Math. Geol.14 (1982), 661\u2013667.","DOI":"10.1007\/BF01033886"},{"key":"2025120523263527558_j_jisys-2015-0031_ref_026_w2aab3b7c79b1b6b1ab2ac26Aa","unstructured":"K. O\u2019Shea, Z. Bandar, and K. Crockett, A novel approach for constructing conversational agents using sentence similarity measures, in: Proceedings of the World Congress on Engineering, vol. 1, 2008."},{"key":"2025120523263527558_j_jisys-2015-0031_ref_027_w2aab3b7c79b1b6b1ab2ac27Aa","unstructured":"E. B. Page, The imminence of grading essays by computer, Phi Delta Kappan47 (1966), 238\u2013243."},{"key":"2025120523263527558_j_jisys-2015-0031_ref_028_w2aab3b7c79b1b6b1ab2ac28Aa","doi-asserted-by":"crossref","unstructured":"K. Papineni, S. Roukos, T. Ward, and W. J. Zhu, BLEU: a method for automatic evaluation of machine translation, in: Proceedings of the 40th Annual Meeting on Association for Computational Linguistics, pp. 311\u2013318, Association for Computational Linguistics, 2002.","DOI":"10.3115\/1073083.1073135"},{"key":"2025120523263527558_j_jisys-2015-0031_ref_029_w2aab3b7c79b1b6b1ab2ac29Aa","unstructured":"D. P\u00e9rez, O. Postolache, E. Alfonseca, D. Cristea, and P. Rodriguez, About the effects of using anaphora resolution in assessing free-text student answers, in: Proceedings of RANLP-2005, 380\u2013386, 2005."},{"key":"2025120523263527558_j_jisys-2015-0031_ref_030_w2aab3b7c79b1b6b1ab2ac30Aa","unstructured":"M. Ramamurthy and I. Krishnamurthi, Parts of speech based sentence similarity computation measures, Int. J. App. Eng. Res.10 (2015), 20176\u201320184."},{"key":"2025120523263527558_j_jisys-2015-0031_ref_031_w2aab3b7c79b1b6b1ab2ac31Aa","unstructured":"G. Salton and M. J. McGill, Introduction to Modern Information Retrieval, McGraw Hill Book Co., New York, 1983."},{"key":"2025120523263527558_j_jisys-2015-0031_ref_032_w2aab3b7c79b1b6b1ab2ac32Aa","unstructured":"S. Saxena, and P. R. Gupta, Automatic assessment of short text answers from computer science domain through pattern based information extraction, in: Proceeding of ASCNT, 109\u2013118, 2009."},{"key":"2025120523263527558_j_jisys-2015-0031_ref_033_w2aab3b7c79b1b6b1ab2ac33Aa","doi-asserted-by":"crossref","unstructured":"T. W. Schoenharl and G. Madey, Evaluation of measurement techniques for the validation of agent-based simulations against streaming data, in: Computational Science, ICCS 2008, pp. 6\u201315, Springer, Berlin, Heidelberg, 2008.","DOI":"10.1007\/978-3-540-69389-5_3"},{"key":"2025120523263527558_j_jisys-2015-0031_ref_034_w2aab3b7c79b1b6b1ab2ac34Aa","doi-asserted-by":"crossref","unstructured":"F. Sebastiani, Machine learning in automated text categorization, ACM Comput. Surv. (CSUR)34 (2002), 1\u201347.","DOI":"10.1145\/505282.505283"},{"key":"2025120523263527558_j_jisys-2015-0031_ref_035_w2aab3b7c79b1b6b1ab2ac35Aa","doi-asserted-by":"crossref","unstructured":"J. Shan, Z. Liu, and W. Zhou, Sentence similarity measure based on events and content words, Fuzzy Systems and Knowledge Discovery7 (2009), 623\u2013627.","DOI":"10.1109\/FSKD.2009.926"},{"key":"2025120523263527558_j_jisys-2015-0031_ref_036_w2aab3b7c79b1b6b1ab2ac36Aa","unstructured":"R. Siddiqi, and C. J. Harrison, On the automated assessment of short free-text responses, in: IAEA Conference Paper, 2008."},{"key":"2025120523263527558_j_jisys-2015-0031_ref_037_w2aab3b7c79b1b6b1ab2ac37Aa","unstructured":"R. Silipo, I. Adae, A. Hart and M. Berthold, Seven Techniques for Dimensionality Reduction. www.knime.org\/files\/knime_seventechniquesdatadimreduction.pdf, 2014."},{"key":"2025120523263527558_j_jisys-2015-0031_ref_038_w2aab3b7c79b1b6b1ab2ac38Aa","doi-asserted-by":"crossref","unstructured":"A. Strehl and J. Ghosh, Value-based customer grouping from large retail data sets, in: AeroSense 2000, pp. 33\u201342, International Society for Optics and Photonics, Bellingham, WA, 2000.","DOI":"10.1117\/12.381756"},{"key":"2025120523263527558_j_jisys-2015-0031_ref_039_w2aab3b7c79b1b6b1ab2ac39Aa","unstructured":"P. N. Tan, M. Steinbach and V. Kumar, Introduction to data mining, Vol. 1, Pearson Addison Wesley, Boston, 2006."},{"key":"2025120523263527558_j_jisys-2015-0031_ref_040_w2aab3b7c79b1b6b1ab2ac40Aa","doi-asserted-by":"crossref","unstructured":"Z. Wu and M. Palmer, Verbs semantics and lexical selection. in: Proceedings of the 32nd annual meeting on Association for Computational Linguistics, pp. 133\u2013138, Association for Computational Linguistics, Stroudsberg, PA, USA, 1994.","DOI":"10.3115\/981732.981751"},{"key":"2025120523263527558_j_jisys-2015-0031_ref_041_w2aab3b7c79b1b6b1ab2ac41Aa","unstructured":"D. Yang, and D. M. Powers, Measuring semantic similarity in the taxonomy of WordNet, in: Proceedings of the Twenty-eighth Australasian conference on Computer Science, vol. 38, pp. 315\u2013322, Australian Computer Society, Inc, 2005."}],"container-title":["Journal of Intelligent Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.degruyter.com\/view\/journals\/jisys\/26\/2\/article-p243.xml","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.degruyterbrill.com\/document\/doi\/10.1515\/jisys-2015-0031\/xml","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.degruyterbrill.com\/document\/doi\/10.1515\/jisys-2015-0031\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,12,5]],"date-time":"2025-12-05T23:26:47Z","timestamp":1764977207000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.degruyterbrill.com\/document\/doi\/10.1515\/jisys-2015-0031\/html"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2016,5,11]]},"references-count":41,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2016,5,11]]},"published-print":{"date-parts":[[2017,4,1]]}},"alternative-id":["10.1515\/jisys-2015-0031"],"URL":"https:\/\/doi.org\/10.1515\/jisys-2015-0031","relation":{},"ISSN":["2191-026X","0334-1860"],"issn-type":[{"type":"electronic","value":"2191-026X"},{"type":"print","value":"0334-1860"}],"subject":[],"published":{"date-parts":[[2016,5,11]]}}}