{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,17]],"date-time":"2026-10-17T05:31:08Z","timestamp":1792215068902,"version":"4.4.0"},"reference-count":54,"publisher":"Association for Computing Machinery (ACM)","issue":"4","license":[{"start":{"date-parts":[[2018,1,31]],"date-time":"2018-01-31T00:00:00Z","timestamp":1517356800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"Australian Research Council (ARC) Linkage Project","award":["LP140100937"],"award-info":[{"award-number":["LP140100937"]}]},{"name":"Youth Innovation Promotion Association of CAS","award":["2014299"],"award-info":[{"award-number":["2014299"]}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"crossref","award":["61672483, U1605251, 61325010 and 61727809"],"award-info":[{"award-number":["61672483, U1605251, 61325010 and 61727809"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Intell. Syst. Technol."],"published-print":{"date-parts":[[2018,7,31]]},"abstract":"<jats:p>\n            Recent decades have witnessed the rapid growth of educational data mining (EDM), which aims at automatically extracting valuable information from large repositories of data generated by or related to people\u2019s learning activities in educational settings. One of the key EDM tasks is cognitive modelling with examination data, and cognitive modelling tries to profile examinees by discovering their latent knowledge state and cognitive level (e.g. the proficiency of specific skills). However, to the best of our knowledge, the problem of extracting information from both objective and subjective examination problems to achieve more precise and interpretable cognitive analysis remains underexplored. To this end, we propose a\n            <jats:italic>fuzzy cognitive diagnosis framework<\/jats:italic>\n            (FuzzyCDF) for examinees\u2019 cognitive modelling with both objective and subjective problems. Specifically, to handle the partially correct responses on subjective problems, we first fuzzify the skill proficiency of examinees. Then we combine fuzzy set theory and educational hypotheses to model the examinees\u2019 mastery on the problems based on their skill proficiency. Finally, we simulate the generation of examination score on each problem by considering slip and guess factors. In this way, the whole diagnosis framework is built. For further comprehensive verification, we apply our FuzzyCDF to three classical cognitive assessment tasks, i.e., predicting examinee performance, slip and guess detection, and cognitive diagnosis visualization. Extensive experiments on three real-world datasets for these assessment tasks prove that FuzzyCDF can reveal the knowledge states and cognitive level of the examinees effectively and interpretatively.\n          <\/jats:p>","DOI":"10.1145\/3168361","type":"journal-article","created":{"date-parts":[[2018,2,1]],"date-time":"2018-02-01T13:10:52Z","timestamp":1517490652000},"page":"1-26","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":148,"title":["Fuzzy Cognitive Diagnosis for Modelling Examinee Performance"],"prefix":"10.1145","volume":"9","author":[{"given":"Qi","family":"Liu","sequence":"first","affiliation":[{"name":"University of Science and Technology of China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6986-5825","authenticated-orcid":false,"given":"Runze","family":"Wu","sequence":"additional","affiliation":[{"name":"University of Science and Technology of China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Enhong","family":"Chen","sequence":"additional","affiliation":[{"name":"University of Science and Technology of China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Guandong","family":"Xu","sequence":"additional","affiliation":[{"name":"University of Technology, Sydney"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yu","family":"Su","sequence":"additional","affiliation":[{"name":"Anhui University 8 IFLYTEK Co., Ltd."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Zhigang","family":"Chen","sequence":"additional","affiliation":[{"name":"IFLYTEK Co., Ltd."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Guoping","family":"Hu","sequence":"additional","affiliation":[{"name":"IFLYTEK Co., Ltd."}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2018,1,31]]},"reference":[{"key":"e_1_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1145\/2566486.2568042"},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1145\/985692.985741"},{"key":"e_1_2_1_3_1","volume-title":"Beck and Beverly Park Woolf","author":"Joseph","year":"2000","unstructured":"Joseph E. Beck and Beverly Park Woolf . 2000 . High-level student modeling with machine learning. In International Conference on Intelligent Tutoring Systems. Springer , 584--593. Joseph E. Beck and Beverly Park Woolf. 2000. High-level student modeling with machine learning. In International Conference on Intelligent Tutoring Systems. Springer, 584--593."},{"key":"e_1_2_1_4_1","volume-title":"Proceedings of the International Workshop on Intelligent Support for Exploratory Environment (EC-TEL\u201908)","volume":"8","author":"Ben-Naim Dror","year":"2008","unstructured":"Dror Ben-Naim , Nadine Marcus , and Mike Bain . 2008 . Visualization and analysis of student interaction in an adaptive exploratory learning environment . In Proceedings of the International Workshop on Intelligent Support for Exploratory Environment (EC-TEL\u201908) , Vol. 8 . Dror Ben-Naim, Nadine Marcus, and Mike Bain. 2008. Visualization and analysis of student interaction in an adaptive exploratory learning environment. In Proceedings of the International Workshop on Intelligent Support for Exploratory Environment (EC-TEL\u201908), Vol. 8."},{"key":"e_1_2_1_5_1","volume-title":"Statistical Theories of Mental Test Scores, Frederic M","author":"Birnbaum Allan","unstructured":"Allan Birnbaum . 1968. Some latent trait models and their use in inferring an examinee\u2019s ability . In Statistical Theories of Mental Test Scores, Frederic M . Lord and Melvin Robert Novick (Eds.). Addison-Wesley , Reading, MA . Allan Birnbaum. 1968. Some latent trait models and their use in inferring an examinee\u2019s ability. In Statistical Theories of Mental Test Scores, Frederic M. Lord and Melvin Robert Novick (Eds.). Addison-Wesley, Reading, MA."},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1080\/10618600.1998.10474787"},{"key":"e_1_2_1_7_1","volume-title":"Redfield","author":"Burns Hugh","year":"2014","unstructured":"Hugh Burns , Carol A. Luckhardt , James W. Parlett , and Carol L . Redfield . 2014 . Intelligent Tutoring Systems: Evolutions in Design. Psychology Press . Hugh Burns, Carol A. Luckhardt, James W. Parlett, and Carol L. Redfield. 2014. Intelligent Tutoring Systems: Evolutions in Design. Psychology Press."},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.2307\/1165298"},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1007\/11774303_17"},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1145\/3132847.3132929"},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1109\/TFUZZ.2014.2310242"},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1007\/BF01099821"},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-540-69132-7_44"},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11336-011-9207-7"},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1007\/BF02295640"},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1177\/0146621610377081"},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1145\/2207243.2207248"},{"key":"e_1_2_1_19_1","volume-title":"Handbook of statistics: Psychometrics, Calyampudi Radhakrishna Rao and Sinharay Sandip (Eds.).","author":"DiBello Louis V.","unstructured":"Louis V. DiBello , Louis A. Roussos , and William Stout . 2006. Review of cognitively diagnostic assessment and a summary of psychometric models . In Handbook of statistics: Psychometrics, Calyampudi Radhakrishna Rao and Sinharay Sandip (Eds.). Vol. 26 . Elsevier , 979--1030. Louis V. DiBello, Louis A. Roussos, and William Stout. 2006. Review of cognitively diagnostic assessment and a summary of psychometric models. In Handbook of statistics: Psychometrics, Calyampudi Radhakrishna Rao and Sinharay Sandip (Eds.). Vol. 26. Elsevier, 979--1030."},{"key":"e_1_2_1_20_1","volume-title":"Proceedings of the NIPS Workshop on Data Driven Education.","author":"Pel\u00e1ez Jorge D\u00edez","year":"2013","unstructured":"Jorge D\u00edez Pel\u00e1ez , \u00d3scar Luaces Rodr\u00edguez , Amparo Alonso Betanzos , Alicia Troncoso , and Antonio Bahamonde Rionda . 2013 . Peer assessment in MOOCs using preference learning via matrix factorization . In Proceedings of the NIPS Workshop on Data Driven Education. Jorge D\u00edez Pel\u00e1ez, \u00d3scar Luaces Rodr\u00edguez, Amparo Alonso Betanzos, Alicia Troncoso, and Antonio Bahamonde Rionda. 2013. Peer assessment in MOOCs using preference learning via matrix factorization. In Proceedings of the NIPS Workshop on Data Driven Education."},{"key":"e_1_2_1_21_1","volume-title":"Multicomponent latent trait models for test design. Test Design: Developments in Psychology and Psychometrics","author":"Embretson Susan E.","year":"1985","unstructured":"Susan E. Embretson . 1985. Multicomponent latent trait models for test design. Test Design: Developments in Psychology and Psychometrics ( 1985 ), 195--218. Susan E. Embretson. 1985. Multicomponent latent trait models for test design. Test Design: Developments in Psychology and Psychometrics (1985), 195--218."},{"key":"e_1_2_1_22_1","volume-title":"Reise","author":"Embretson Susan E.","year":"2013","unstructured":"Susan E. Embretson and Steven P . Reise . 2013 . Item Response Theory for Psychologists. Psychology Press . Susan E. Embretson and Steven P. Reise. 2013. Item Response Theory for Psychologists. Psychology Press."},{"key":"e_1_2_1_23_1","volume-title":"Fuzzy sets and fuzzy logic: Theory and applications","author":"Klir George","unstructured":"George Klir and Bo Yuan . 1995. Fuzzy sets and fuzzy logic: Theory and applications . Englewood Cliffs, NJ : Prentice-Hall . George Klir and Bo Yuan. 1995. Fuzzy sets and fuzzy logic: Theory and applications. Englewood Cliffs, NJ: Prentice-Hall."},{"key":"e_1_2_1_24_1","volume-title":"Markov chain Monte Carlo. Encyclopedia of Biostatistics","author":"Gilks Walter R.","unstructured":"Walter R. Gilks . 2005. Markov chain Monte Carlo. Encyclopedia of Biostatistics . Walter R. Gilks. 2005. Markov chain Monte Carlo. Encyclopedia of Biostatistics."},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-07221-0_92"},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1177\/014662168400800311"},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1080\/0013188840260108"},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.5555\/648031.761136"},{"key":"e_1_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.1093\/biomet\/57.1.97"},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1111\/jedm.12036"},{"key":"e_1_2_1_31_1","doi-asserted-by":"crossref","unstructured":"Zhenya Huang Qi Liu Enhong Chen Hongke Zhao Mingyong Gao Si Wei Yu Su and Guoping Hu. 2017. Question difficulty prediction for reading problems in standard tests. In AAAI. 1352--1359.  Zhenya Huang Qi Liu Enhong Chen Hongke Zhao Mingyong Gao Si Wei Yu Su and Guoping Hu. 2017. Question difficulty prediction for reading problems in standard tests. In AAAI. 1352--1359.","DOI":"10.1609\/aaai.v31i1.10740"},{"key":"e_1_2_1_32_1","volume-title":"Parsons","author":"Hulin Charles Lee","year":"1983","unstructured":"Charles Lee Hulin , Fritz Drasgow , and Charles K . Parsons . 1983 . Item Response Theory: Application to Psychological Measurement. Dorsey Press . Charles Lee Hulin, Fritz Drasgow, and Charles K. Parsons. 1983. Item Response Theory: Application to Psychological Measurement. Dorsey Press."},{"key":"e_1_2_1_34_1","volume-title":"Foundations of Fuzzy Control: A Practical Approach","author":"Jantzen Jan","unstructured":"Jan Jantzen . 2013. Foundations of Fuzzy Control: A Practical Approach . John Wiley 8 Sons. Jan Jantzen. 2013. Foundations of Fuzzy Control: A Practical Approach. John Wiley 8 Sons."},{"key":"e_1_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.1177\/01466210122032064"},{"key":"e_1_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.1109\/MC.2009.263"},{"key":"e_1_2_1_37_1","unstructured":"Daniel D. Lee and H. Sebastian Seung. 2001. Algorithms for non-negative matrix factorization. In Advances in NIPS. 556--562.  Daniel D. Lee and H. Sebastian Seung. 2001. Algorithms for non-negative matrix factorization. In Advances in NIPS. 556--562."},{"key":"e_1_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.1145\/2330601.2330642"},{"key":"e_1_2_1_39_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICDM.2015.78"},{"key":"e_1_2_1_40_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-32049-6_24"},{"key":"e_1_2_1_41_1","volume-title":"Jansen","author":"Liu Zhe","year":"2015","unstructured":"Zhe Liu and Bernard J . Jansen . 2015 . Subjective versus objective questions: Perception of question subjectivity in social Q8A. In Proceedings of the International Conference on Social Computing, Behavioral-Cultural Modeling, and Prediction. Springer , 131--140. Zhe Liu and Bernard J. Jansen. 2015. Subjective versus objective questions: Perception of question subjectivity in social Q8A. In Proceedings of the International Conference on Social Computing, Behavioral-Cultural Modeling, and Prediction. Springer, 131--140."},{"key":"e_1_2_1_42_1","doi-asserted-by":"publisher","DOI":"10.1145\/347090.347184"},{"key":"e_1_2_1_43_1","unstructured":"Andriy Mnih and Ruslan Salakhutdinov. 2007. Probabilistic matrix factorization. In Advances in NIPS. 1257--1264.   Andriy Mnih and Ruslan Salakhutdinov. 2007. Probabilistic matrix factorization. In Advances in NIPS. 1257--1264."},{"key":"e_1_2_1_44_1","volume-title":"Brennan","author":"Nichols Paul D.","year":"2012","unstructured":"Paul D. Nichols , Susan F. Chipman , and Robert L . Brennan . 2012 . Cognitively Diagnostic Assessment. Routledge . Paul D. Nichols, Susan F. Chipman, and Robert L. Brennan. 2012. Cognitively Diagnostic Assessment. Routledge."},{"key":"e_1_2_1_45_1","volume-title":"Proceedings of the 1st International Conference on Educational Data Mining. Citeseer.","author":"Pardos Zachary A.","unstructured":"Zachary A. Pardos , Joseph E. Beck , Carolina Ruiz , and Neil T. Heffernan . 2008. The composition effect: Conjunctive or compensatory? An analysis of multi-skill math questions in ITS . In Proceedings of the 1st International Conference on Educational Data Mining. Citeseer. Zachary A. Pardos, Joseph E. Beck, Carolina Ruiz, and Neil T. Heffernan. 2008. The composition effect: Conjunctive or compensatory? An analysis of multi-skill math questions in ITS. In Proceedings of the 1st International Conference on Educational Data Mining. Citeseer."},{"key":"e_1_2_1_46_1","volume-title":"Proceedings of the 4th Berkeley Symposium on Mathematical Statistics and Probability","volume":"4","author":"Rasch Georg","year":"1961","unstructured":"Georg Rasch . 1961 . On general laws and the meaning of measurement in psychology . In Proceedings of the 4th Berkeley Symposium on Mathematical Statistics and Probability , Vol. 4 . University of California Press, Berkeley, CA, 321--333. Georg Rasch. 1961. On general laws and the meaning of measurement in psychology. In Proceedings of the 4th Berkeley Symposium on Mathematical Statistics and Probability, Vol. 4. University of California Press, Berkeley, CA, 321--333."},{"key":"e_1_2_1_47_1","volume-title":"Education: Theory and Practice","author":"Roussos L.","year":"2006","unstructured":"L. Roussos , L. DiBello , W. Stout , S. M. Hartz , R. A. Henson , and J. L. Templin . 2006 . Diagnostic skills-based testing using the fusion-model-based arpeggio system. In Cognitively Diagnostic Assessment in Education: Theory and Practice , Jacqueline Leighton and Gierl Mark (Eds.). Cambridge University Press . L. Roussos, L. DiBello, W. Stout, S. M. Hartz, R. A. Henson, and J. L. Templin. 2006. Diagnostic skills-based testing using the fusion-model-based arpeggio system. In Cognitively Diagnostic Assessment in Education: Theory and Practice, Jacqueline Leighton and Gierl Mark (Eds.). Cambridge University Press."},{"key":"e_1_2_1_48_1","volume-title":"Psychometrika Monograph. 18, 34 (4, Pt. 2).","author":"Samejima Fumiko","unstructured":"Fumiko Samejima . 1972. A general model for free-response data . In Psychometrika Monograph. 18, 34 (4, Pt. 2). Fumiko Samejima. 1972. A general model for free-response data. In Psychometrika Monograph. 18, 34 (4, Pt. 2)."},{"key":"e_1_2_1_49_1","volume-title":"Proceedings of the 7th Int\u2019l Conf. on Educational Data Mining. 14--20","author":"Sun Yuan","year":"2014","unstructured":"Yuan Sun , Shiwei Ye , Shunya Inoue , and Yi Sun . 2014 . Alternating recursive method for Q-matrix learning . In Proceedings of the 7th Int\u2019l Conf. on Educational Data Mining. 14--20 . Yuan Sun, Shiwei Ye, Shunya Inoue, and Yi Sun. 2014. Alternating recursive method for Q-matrix learning. In Proceedings of the 7th Int\u2019l Conf. on Educational Data Mining. 14--20."},{"key":"e_1_2_1_50_1","volume-title":"Analysis of errors in fraction addition and subtraction problems (final report for grant no. NIE-G-81-0002)","author":"Tatsuoka K. K.","year":"1984","unstructured":"K. K. Tatsuoka . 1984. Analysis of errors in fraction addition and subtraction problems (final report for grant no. NIE-G-81-0002) . University of Illinois--Urbana . Computer-based Education Research Laboratory ( 1984 ). K. K. Tatsuoka. 1984. Analysis of errors in fraction addition and subtraction problems (final report for grant no. NIE-G-81-0002). University of Illinois--Urbana. Computer-based Education Research Laboratory (1984)."},{"key":"e_1_2_1_51_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.procs.2010.08.006"},{"key":"e_1_2_1_52_1","doi-asserted-by":"publisher","DOI":"10.1109\/KSE.2015.9"},{"key":"e_1_2_1_53_1","volume-title":"Proceedings of the KDD 2010 Cup 2010 Workshop: Knowledge Discovery in Educational Data. 13--23","author":"Toscher A.","year":"2010","unstructured":"A. Toscher and Michael Jahrer . 2010 . Collaborative filtering applied to educational data mining . In Proceedings of the KDD 2010 Cup 2010 Workshop: Knowledge Discovery in Educational Data. 13--23 . A. Toscher and Michael Jahrer. 2010. Collaborative filtering applied to educational data mining. In Proceedings of the KDD 2010 Cup 2010 Workshop: Knowledge Discovery in Educational Data. 13--23."},{"key":"e_1_2_1_54_1","volume-title":"Proceedings of the 24th International Joint Conference on Artificial Intelligence. AAAI Press, 1017--1024","author":"Wu Runze","year":"2015","unstructured":"Runze Wu , Qi Liu , Yuping Liu , Enhong Chen , Yu Su , Zhigang Chen , and Guoping Hu . 2015 . Cognitive modelling for predicting examinee performance . In Proceedings of the 24th International Joint Conference on Artificial Intelligence. AAAI Press, 1017--1024 . Runze Wu, Qi Liu, Yuping Liu, Enhong Chen, Yu Su, Zhigang Chen, and Guoping Hu. 2015. Cognitive modelling for predicting examinee performance. In Proceedings of the 24th International Joint Conference on Artificial Intelligence. AAAI Press, 1017--1024."},{"key":"e_1_2_1_55_1","doi-asserted-by":"publisher","DOI":"10.1145\/3041021.3054156"},{"key":"e_1_2_1_56_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0019-9958(65)90241-X"}],"container-title":["ACM Transactions on Intelligent Systems and Technology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3168361","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3168361","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T02:26:08Z","timestamp":1750213568000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3168361"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2018,1,31]]},"references-count":54,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2018,7,31]]}},"alternative-id":["10.1145\/3168361"],"URL":"https:\/\/doi.org\/10.1145\/3168361","relation":{},"ISSN":["2157-6904","2157-6912"],"issn-type":[{"value":"2157-6904","type":"print"},{"value":"2157-6912","type":"electronic"}],"subject":[],"published":{"date-parts":[[2018,1,31]]},"assertion":[{"value":"2016-12-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2017-11-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2018-01-31","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}