{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,12]],"date-time":"2026-01-12T04:37:22Z","timestamp":1768192642999,"version":"3.49.0"},"reference-count":36,"publisher":"MDPI AG","issue":"5","license":[{"start":{"date-parts":[[2017,4,25]],"date-time":"2017-04-25T00:00:00Z","timestamp":1493078400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Symmetry"],"abstract":"<jats:p>The setting of standards is a critical process in educational evaluation, but it is time-consuming and expensive because it is generally conducted by an education experts group. The purpose of this paper is to find a suitable cluster validity index that considers the futures of item response data for setting cut-off scores. In this study, nine representative cluster validity indexes were used to evaluate the clustering results. Cohen\u2019s kappa coefficient is used to check the conformity between a set cut-off score using four clustering techniques and a cut-off score set by experts. We compared the cut-off scores by each cluster validity index and by a group of experts. The experimental results show that the entropy-based method considers the features of item response data, so it has a realistic possibility of applying a clustering evaluation method to the setting of standards in criterion referenced evaluation.<\/jats:p>","DOI":"10.3390\/sym9050062","type":"journal-article","created":{"date-parts":[[2017,4,25]],"date-time":"2017-04-25T13:21:12Z","timestamp":1493126472000},"page":"62","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":14,"title":["Analysis of Clustering Evaluation Considering Features of Item Response Data Using Data Mining Technique for Setting Cut-Off Scores"],"prefix":"10.3390","volume":"9","author":[{"given":"Byoungwook","family":"Kim","sequence":"first","affiliation":[{"name":"Creative Informatics &amp; Computing Institute, Korea University, Seoul 02841, Korea"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"JaMee","family":"Kim","sequence":"additional","affiliation":[{"name":"Major of Computer Science Education, Graduate School of Education, Korea University, Seoul 02841, Korea"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Gangman","family":"Yi","sequence":"additional","affiliation":[{"name":"Department of Multimedia Engineering, Dongguk University, Seoul 04620, Korea"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2017,4,25]]},"reference":[{"key":"ref_1","unstructured":"Nitko, A.J. (2001). Educational Assessment of Students, Merrill\/Prentice Hall. [3rd ed.]."},{"key":"ref_2","first-page":"120","article-title":"Norm- and criterion-referenced testing","volume":"5","author":"Bond","year":"1996","journal-title":"Pract. Assess. Res. Eval."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"7","DOI":"10.1111\/j.1745-3992.1991.tb00171.x","article-title":"Relevant classroom assessment training for teachers","volume":"10","author":"Stiggins","year":"1991","journal-title":"Educ. Meas. Issues Pract."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"31","DOI":"10.1111\/j.1745-3992.2004.tb00166.x","article-title":"Setting performance standards: contemporary mMethods","volume":"23","author":"Cizek","year":"2004","journal-title":"Educ. Meas. Issues Pract."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"329","DOI":"10.1109\/TE.2003.813516","article-title":"A test-sheet-generating algorithm for multiple assessment requirements","volume":"46","author":"Hwang","year":"2003","journal-title":"IEEE Trans. Educ."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"264","DOI":"10.1145\/331499.331504","article-title":"Data clustering: A review","volume":"31","author":"Jain","year":"1999","journal-title":"ACM Comput. Surv."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"297","DOI":"10.1111\/j.1745-3984.1978.tb00077.x","article-title":"As always, provocative","volume":"15","author":"Popham","year":"1978","journal-title":"J. Educ. Meas."},{"key":"ref_8","first-page":"36","article-title":"The bookmark procedure for setting cut-scores and finalizing performance standards strengths and weaknesses","volume":"52","author":"Lin","year":"2006","journal-title":"Alta. J. Educ. Res."},{"key":"ref_9","unstructured":"Brennan, R.L. (2006). Setting Performance Standards. Educational Measurement, Greenwood."},{"key":"ref_10","unstructured":"Morgan, D., and Perie, M. (2004). Setting Standards in Education: Choosing the Best Method for Your Assessment and Population, Educational Testing Service (ETS)."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"183","DOI":"10.1007\/978-3-540-71974-8_8","article-title":"Applying data mining techniques to e-Learning problems","volume":"62","author":"Castro","year":"2007","journal-title":"Stud. Comput. Intell."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"464","DOI":"10.1046\/j.1365-2923.2003.01495.x","article-title":"Setting standards on educational tests","volume":"37","author":"Norcini","year":"2003","journal-title":"Med. Educ."},{"key":"ref_13","first-page":"1","article-title":"Recommending cut scores with a subset of items: An empirical illustration","volume":"15","author":"Buckendahl","year":"2010","journal-title":"Pract. Assess. Res. Eval."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"107","DOI":"10.1023\/A:1012801612483","article-title":"On clustering validation techniques","volume":"17","author":"Halkidi","year":"2001","journal-title":"J. Intell. Inf. Syst."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Kim, H.C., and Kwak, E.Y. (2005). Information-Based Pruning for Interesting Association Rule Mining in the Item Response Dataset, Springer. Lecture Notes in Computer Science.","DOI":"10.1007\/11552413_54"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"320","DOI":"10.1119\/1.11836","article-title":"Entropy state of a multiple choice examination and the evaluation of understanding","volume":"47","author":"Vawter","year":"1979","journal-title":"Am. J. Phys."},{"key":"ref_17","first-page":"292","article-title":"Application of k-means clustering algorithm for prediction of students\u2019 academic performance","volume":"7","author":"Oyelade","year":"2010","journal-title":"Int. J. Comput. Sci. Inf. Secur."},{"key":"ref_18","first-page":"24","article-title":"Data mining model for higher education system","volume":"43","author":"Ayesha","year":"2010","journal-title":"Eur. J. Sci. Res."},{"key":"ref_19","unstructured":"Sacin, C.V., Agapito, J.B., Shafti, L., and Ortigosa, A. (2009, January 1\u20133). Recommendation in higher education using data mining techniques. Proceedings of the 2nd International Conference on Educational Data Mining, Cordoba, Spain."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"560","DOI":"10.3923\/itj.2006.560.564","article-title":"Data mining model for a better higher educational system","volume":"5","author":"Shyamala","year":"2006","journal-title":"Inf. Technol. J."},{"key":"ref_21","unstructured":"Sembiring, S., Zarlis, M., Hartama, D., Ramliana, S., and Wani, E. (2011, January 1\u20133). Prediction of student academic performance by an application of data mining techniques. Proceedings of the 2011 International Conference on Management and Artificial Intelligence, Bali, Indonesia."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"139","DOI":"10.1007\/BF00114265","article-title":"Knowledge acquisition via incremental conceptual clustering","volume":"2","author":"Fisher","year":"1987","journal-title":"Mach. Learn."},{"key":"ref_23","unstructured":"Bing, L. (2006). Web Data Mining: Exploring Hyperlinks, Contents, and Usage Data (Data-Centric Systems and Applications), Springer."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"581","DOI":"10.1613\/jair.3003","article-title":"Which clustering do you want? Inducing your ideal clustering with minimal feedback","volume":"39","author":"Dasgupta","year":"2010","journal-title":"J. Artif. Intell. Res."},{"key":"ref_25","unstructured":"Crabtree, D., Gao, X., and Andreae, P. (2005, January 19\u201322). Standardized evaluation method for web clustering results. Proceedings of the 2005 IEEE\/WIC\/ACM International Conference on Web Intelligence, Compiegne, France."},{"key":"ref_26","first-page":"151","article-title":"A grading method for student\u2019s achievements based on the clustering technique","volume":"12","author":"Park","year":"2002","journal-title":"Fuzzy Log. Intell. Syst."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"355","DOI":"10.1007\/BF00962924","article-title":"Neighborhood and associative query answering","volume":"1","author":"Chu","year":"1992","journal-title":"J. Intell. Inf. Syst."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"343","DOI":"10.1007\/BF00116838","article-title":"Conceptual clustering, categorization, and polymorphy","volume":"3","author":"Hanson","year":"1989","journal-title":"Mach. Learn."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"990","DOI":"10.1016\/j.compedu.2010.11.003","article-title":"Measuring achievement of ICT competency for students in Korea","volume":"56","author":"Cha","year":"2011","journal-title":"Comput. Educ."},{"key":"ref_30","unstructured":"Crocker, L., and Algina, J. (1986). Introduction to Classical & Modern Test Theory, Holt, Rinehart and Winston."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"37","DOI":"10.1177\/001316446002000104","article-title":"A coefficient of agreement for nominal scales","volume":"20","author":"Cohen","year":"1960","journal-title":"Educ. Psychol. Meas."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"159","DOI":"10.2307\/2529310","article-title":"The measurement of observer agreement for categorical data","volume":"33","author":"Landis","year":"1977","journal-title":"Biometrics"},{"key":"ref_33","unstructured":"Baker, F.B., and Kim, S.-H. (2004). Item Response Theory: Parameter Estimation Techniques, Marcel Dekker. [2nd ed.]."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"95","DOI":"10.1162\/089120104773633402","article-title":"The kappa statistic: A second look","volume":"30","author":"DiEugenio","year":"2004","journal-title":"Comput. Linguist."},{"key":"ref_35","first-page":"1","article-title":"Processing and analysing large log data files of a virtual campus","volume":"3","author":"Xhafa","year":"2012","journal-title":"J. Converg."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"379","DOI":"10.1002\/j.1538-7305.1948.tb01338.x","article-title":"A mathematical theory of communication","volume":"27","author":"Shannon","year":"1948","journal-title":"Bell Syst. Tech. J."}],"container-title":["Symmetry"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2073-8994\/9\/5\/62\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T18:33:44Z","timestamp":1760207624000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2073-8994\/9\/5\/62"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2017,4,25]]},"references-count":36,"journal-issue":{"issue":"5","published-online":{"date-parts":[[2017,5]]}},"alternative-id":["sym9050062"],"URL":"https:\/\/doi.org\/10.3390\/sym9050062","relation":{},"ISSN":["2073-8994"],"issn-type":[{"value":"2073-8994","type":"electronic"}],"subject":[],"published":{"date-parts":[[2017,4,25]]}}}