{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,11,25]],"date-time":"2025-11-25T08:50:21Z","timestamp":1764060621589,"version":"3.41.0"},"reference-count":23,"publisher":"Association for Computing Machinery (ACM)","issue":"May","license":[{"start":{"date-parts":[[2014,5,1]],"date-time":"2014-05-01T00:00:00Z","timestamp":1398902400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Ubiquity"],"published-print":{"date-parts":[[2014,5]]},"abstract":"<jats:p>Advanced educational technologies are developing rapidly and online MOOC courses are becoming more prevalent, creating an enthusiasm for the seemingly limitless data-driven possibilities to affect advances in learning and enhance the learning experience. For these possibilities to unfold, the expertise and collaboration of many specialists will be necessary to improve data collection, to foster the development of better predictive models, and to assure models are interpretable and actionable. The big data collected from MOOCs needs to be bigger, not in its height (number of students) but in its width more meta-data and information on learners' cognitive and self-regulatory states needs to be collected in addition to correctness and completion rates. This more detailed articulation will help open up the black box approach to machine learning models where prediction is the primary goal. Instead, a data-driven learner model approach uses fine grain data that is conceived and developed from cognitive principles to build explanatory models with practical implications to improve student learning.<\/jats:p>","DOI":"10.1145\/2591682","type":"journal-article","created":{"date-parts":[[2014,6,10]],"date-time":"2014-06-10T12:50:17Z","timestamp":1402404617000},"page":"1-13","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":4,"title":["Data-driven Learner Modeling to Understand and Improve Online Learning"],"prefix":"10.1145","volume":"2014","author":[{"given":"Kenneth R.","family":"Koedinger","sequence":"first","affiliation":[{"name":"Carnegie Mellon"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Elizabeth A.","family":"McLaughlin","sequence":"additional","affiliation":[{"name":"Carnegie Mellon University"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"John C.","family":"Stamper","sequence":"additional","affiliation":[{"name":"Carnegie Mellon University and LearnLab's DataShop"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2014,5]]},"reference":[{"key":"e_1_2_1_1_1","volume-title":"Organizing Instruction and Study to Improve Student Learning (NCER 2007--2004)","author":"Pashler H.","year":"2007","unstructured":"Pashler , H. , Bain , P. , Bottge , B. , Graesser , A. , Koedinger , K. , McDaniel , M. , and Metcalfe , J . Organizing Instruction and Study to Improve Student Learning (NCER 2007--2004) . Washington, DC : National Center for Education Research, Institute of Education Sciences, U.S. Department of Education . 2007 . Pashler, H., Bain, P., Bottge, B., Graesser, A., Koedinger, K., McDaniel, M., and Metcalfe, J. Organizing Instruction and Study to Improve Student Learning (NCER 2007--2004). Washington, DC: National Center for Education Research, Institute of Education Sciences, U.S. Department of Education. 2007."},{"key":"e_1_2_1_2_1","volume-title":"Jossey-Bass","author":"Ambrose S. A.","year":"2010","unstructured":"Ambrose , S. A. , Bridges , M. W. , DiPietro , M. , Lovett , M. C. , and Norman , M. K . How Learning Works: Seven Research-Based Principles for Smart Teaching . Jossey-Bass , San Francisco , 2010 . Ambrose, S. A., Bridges, M. W., DiPietro, M., Lovett, M. C., and Norman, M. K. How Learning Works: Seven Research-Based Principles for Smart Teaching. Jossey-Bass, San Francisco, 2010."},{"key":"e_1_2_1_3_1","volume-title":"Opening statement for symposium: Technology to Advance Learning and Learning Research. ACM Ubiquity","author":"Thille C.","year":"2004","unstructured":"Thille , C. Opening statement for symposium: Technology to Advance Learning and Learning Research. ACM Ubiquity April 2004 . Thille, C. Opening statement for symposium: Technology to Advance Learning and Learning Research. ACM Ubiquity April 2004."},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1207\/s15327809jls1302_1"},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1207\/S1532690XCI1802_03"},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.amjsurg.2002.12.005"},{"key":"e_1_2_1_7_1","series-title":"Lecture Notes in Computer Science","volume-title":"Intelligent Tutoring Systems","author":"Lovett M. C.","year":"1998","unstructured":"Lovett , M. C. Cognitive task analysis in service of intelligent tutoring systems design: A case study in statistics . In B. P. Goettl, H. M. Halff, C. L. Redfield, and V. J. Shute (Eds.) Intelligent Tutoring Systems , Lecture Notes in Computer Science Volume 1452 . Springer , New York , 1998 , 234--243. Lovett, M. C. Cognitive task analysis in service of intelligent tutoring systems design: A case study in statistics. In B. P. Goettl, H. M. Halff, C. L. Redfield, and V. J. Shute (Eds.) Intelligent Tutoring Systems, Lecture Notes in Computer Science Volume 1452. Springer, New York, 1998, 234--243."},{"key":"e_1_2_1_8_1","volume-title":"The Open Learning Initiative: Measuring the effectiveness of the OLI learning course in accelerating student learning. Journal of Interactive Media in Education","author":"Lovett M.","year":"2008","unstructured":"Lovett , M. , Meyer , O. , Thille , C. The Open Learning Initiative: Measuring the effectiveness of the OLI learning course in accelerating student learning. Journal of Interactive Media in Education . 2008 . Lovett, M., Meyer, O., Thille, C. The Open Learning Initiative: Measuring the effectiveness of the OLI learning course in accelerating student learning. Journal of Interactive Media in Education. 2008."},{"key":"e_1_2_1_9_1","volume-title":"Effectiveness of Cognitive Tutor Algebra I at Scale. Educational Evaluation and Policy Analysis","author":"Pane J. F.","year":"2013","unstructured":"Pane , J. F. , Griffin , B. A. , McCaffrey , D. F. , Karam , R. Effectiveness of Cognitive Tutor Algebra I at Scale. Educational Evaluation and Policy Analysis . 2013 . Published online 11 November 2013 as doi:10.3102\/0162373713507480. 10.3102\/0162373713507480 Pane, J. F., Griffin, B. A., McCaffrey, D. F., Karam, R. Effectiveness of Cognitive Tutor Algebra I at Scale. Educational Evaluation and Policy Analysis. 2013. Published online 11 November 2013 as doi:10.3102\/0162373713507480."},{"key":"e_1_2_1_10_1","volume-title":"Proceedings of the Twentieth Annual Conference of the Cognitive Science Society. Erlbaum","author":"Heffernan N.","year":"1998","unstructured":"Heffernan , N. and Koedinger , K. R . A developmental model for algebra symbolization: The results of a difficulty factors assessment . In Proceedings of the Twentieth Annual Conference of the Cognitive Science Society. Erlbaum , Hillsdale, NJ , 1998 , 484--489. Heffernan, N. and Koedinger, K. R. A developmental model for algebra symbolization: The results of a difficulty factors assessment. In Proceedings of the Twentieth Annual Conference of the Cognitive Science Society. Erlbaum, Hillsdale, NJ, 1998, 484--489."},{"key":"e_1_2_1_11_1","volume-title":"Proceedings of the 32nd Annual Conference of the Cognitive Science Society. Cognitive Science Society","author":"Koedinger K.R.","year":"2010","unstructured":"Koedinger , K.R. and McLaughlin , E.A. Seeing language learning inside the math: Cognitive analysis yields transfer. In S. Ohlsson and R. Catrambone (Eds.) , Proceedings of the 32nd Annual Conference of the Cognitive Science Society. Cognitive Science Society ( 2010 , Austin). 471--476. Koedinger, K.R. and McLaughlin, E.A. Seeing language learning inside the math: Cognitive analysis yields transfer. In S. Ohlsson and R. Catrambone (Eds.), Proceedings of the 32nd Annual Conference of the Cognitive Science Society. Cognitive Science Society (2010, Austin). 471--476."},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.5555\/2026506.2026554"},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-39112-5_43"},{"key":"e_1_2_1_14_1","first-page":"3","article-title":"The behavior of tutoring systems","volume":"16","author":"VanLehn K","year":"2006","unstructured":"VanLehn , K . The behavior of tutoring systems . International Journal of Artificial Intelligence in Education 16 , 3 ( 2006 ), 227--265. VanLehn, K. The behavior of tutoring systems. International Journal of Artificial Intelligence in Education 16, 3 (2006), 227--265.","journal-title":"International Journal of Artificial Intelligence in Education"},{"key":"e_1_2_1_15_1","volume-title":"Handbook of Educational Data Mining","author":"Koedinger K.R.","year":"2010","unstructured":"Koedinger , K.R. , Baker , R.S.J.D. , Cunningham , K. , Skogsholm , A. , Leber , B. , Stamper , J. A Data Repository for the EDM community: The PSLC DataShop . In Romero, Ventura, Pechenizkiy, Baker, (Eds.) Handbook of Educational Data Mining . CRC Press , 2010 . Koedinger, K.R., Baker, R.S.J.D., Cunningham, K., Skogsholm, A., Leber, B., Stamper, J. A Data Repository for the EDM community: The PSLC DataShop. In Romero, Ventura, Pechenizkiy, Baker, (Eds.) Handbook of Educational Data Mining. CRC Press, 2010."},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1207\/s15516709cog2602_1"},{"key":"e_1_2_1_17_1","volume-title":"Proceedings of the 11th International Conference on Artificial Intelligence in Education, AI-ED 2003","author":"Aleven V.","year":"2003","unstructured":"Aleven V. , Koedinger , K. R. , and Popescu , O . A tutorial dialog system to support self-explanation: Evaluation and open questions. In U. Hoppe, F. Verdejo, and J. Kay (Eds.) , Proceedings of the 11th International Conference on Artificial Intelligence in Education, AI-ED 2003 . IOS Press, Amsterdam 2003 , 39--46. Aleven V., Koedinger, K. R., and Popescu, O. A tutorial dialog system to support self-explanation: Evaluation and open questions. In U. Hoppe, F. Verdejo, and J. Kay (Eds.), Proceedings of the 11th International Conference on Artificial Intelligence in Education, AI-ED 2003. IOS Press, Amsterdam 2003, 39--46."},{"key":"e_1_2_1_18_1","volume-title":"Proceedings of the 35th Annual Conference of the Cognitive Science Society.","author":"Li N.","year":"2013","unstructured":"Li , N. , Stampfer , E. , Cohen , W.W. , and Koedinger , K.R . General and efficient cognitive model discovery using a simulated student. M. Knauff, M. Pauen, N. Sebanz, and I. Wachsmuth (Eds.) . In Proceedings of the 35th Annual Conference of the Cognitive Science Society. 2013 . Li, N., Stampfer, E., Cohen, W.W., and Koedinger, K.R. General and efficient cognitive model discovery using a simulated student. M. Knauff, M. Pauen, N. Sebanz, and I. Wachsmuth (Eds.). In Proceedings of the 35th Annual Conference of the Cognitive Science Society. 2013."},{"key":"e_1_2_1_19_1","volume-title":"Proceedings of the 5th International Conference on Educational Data Mining (June","author":"Koedinger K. R.","year":"2012","unstructured":"Koedinger , K. R. , McLaughlin , E. A. , and Stamper , J. C. Automated Student Model Improvement . Yacef, K., Za\u00efane , O. , Hershkovitz , H. , Yudelson , M. , and Stamper , J. (eds.) Proceedings of the 5th International Conference on Educational Data Mining (June 2012 , Chania , Greece ). Koedinger, K. R., McLaughlin, E. A., and Stamper, J. C. Automated Student Model Improvement. Yacef, K., Za\u00efane, O., Hershkovitz, H., Yudelson, M., and Stamper, J. (eds.) Proceedings of the 5th International Conference on Educational Data Mining (June 2012, Chania, Greece)."},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-540-30139-4_50"},{"key":"e_1_2_1_21_1","unstructured":"Koedinger K. R. Brunskill E. S.J.D. Baker R. McLaughlin E. A. and Stamper J. C. (Submitted). New Potentials for Data-Driven Intelligent Tutoring System Development and Optimization. Artificial Intelligence Magazine.  Koedinger K. R. Brunskill E. S.J.D. Baker R. McLaughlin E. A. and Stamper J. C. (Submitted). New Potentials for Data-Driven Intelligent Tutoring System Development and Optimization. Artificial Intelligence Magazine."},{"key":"e_1_2_1_22_1","volume-title":"Proceedings of the 1st Workshop on Massive Open Online Courses at the 16th Annual Conference on Artificial Intelligence in Education (July","author":"Huang J.","year":"2013","unstructured":"Huang , J. , Piech , C. , Nguyen , A. , and Guibas , L . Syntactic and Functional Variability of a Million Code Submissions in a Machine Learning MOOC . In Proceedings of the 1st Workshop on Massive Open Online Courses at the 16th Annual Conference on Artificial Intelligence in Education (July 2013 , Memphis, TN). Huang, J., Piech, C., Nguyen, A., and Guibas, L. Syntactic and Functional Variability of a Million Code Submissions in a Machine Learning MOOC. In Proceedings of the 1st Workshop on Massive Open Online Courses at the 16th Annual Conference on Artificial Intelligence in Education (July 2013, Memphis, TN)."},{"key":"e_1_2_1_23_1","volume-title":"Tuned Models of Peer Assessment in MOOCs. D'Mello","author":"Piech C.","year":"2013","unstructured":"Piech , C. , Huang , J. , Chen , Z. , Do , C. , Ng , A. , and Koller , D . Tuned Models of Peer Assessment in MOOCs. D'Mello , S. K., Calvo, R. A., and Olney, A. (eds.) In Proceedings of the 6th International Conference on Educational Data Mining ( July 2013 , Memphis, TN) . 153--160). Piech, C., Huang, J., Chen, Z., Do, C., Ng, A., and Koller, D. Tuned Models of Peer Assessment in MOOCs. D'Mello, S. K., Calvo, R. A., and Olney, A. (eds.) In Proceedings of the 6th International Conference on Educational Data Mining (July 2013, Memphis, TN). 153--160)."}],"container-title":["Ubiquity"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/2591682","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/2591682","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T06:55:59Z","timestamp":1750229759000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/2591682"}},"subtitle":["MOOCs and technology to advance learning and learning research (Ubiquity symposium)"],"short-title":[],"issued":{"date-parts":[[2014,5]]},"references-count":23,"journal-issue":{"issue":"May","published-print":{"date-parts":[[2014,5]]}},"alternative-id":["10.1145\/2591682"],"URL":"https:\/\/doi.org\/10.1145\/2591682","relation":{},"ISSN":["1530-2180"],"issn-type":[{"type":"electronic","value":"1530-2180"}],"subject":[],"published":{"date-parts":[[2014,5]]},"assertion":[{"value":"2014-05-01","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}