{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,11]],"date-time":"2026-07-11T13:54:53Z","timestamp":1783778093843,"version":"3.55.0"},"reference-count":41,"publisher":"Springer Science and Business Media LLC","issue":"11","license":[{"start":{"date-parts":[[2023,12,16]],"date-time":"2023-12-16T00:00:00Z","timestamp":1702684800000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2023,12,16]],"date-time":"2023-12-16T00:00:00Z","timestamp":1702684800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"name":"TU\/e BOOST! Program"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Educ Inf Technol"],"published-print":{"date-parts":[[2024,8]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Self-regulated learning has seen a large increase in research interest due to its importance for online learning of higher education students. Several ways to measure self-regulated learning have been suggested. However, most measurements are either obtrusive, necessitating time and effort from students and potentially influencing the learning process, or only partially portable across courses. In the current study, we develop clickstream-based scales of four self-regulated learning phases that we show are portable across courses. The final scales are based on the COPES model and include two strong and reliable dimensions, enactment and adaptation, one dimension that performs reasonably, task definition, and a weaker one, goal-setting. By considering portability as the main criterion in the scale construction process, we ensured reliable transfer to both similar and dissimilar courses. When considering convergent validity, the created scale has higher bivariate and partial correlations with final student grades than the often-used self-reported MSLQ-SRL scale. We discuss limitations and future research to improve the scale further and facilitate adoption.<\/jats:p>","DOI":"10.1007\/s10639-023-12372-6","type":"journal-article","created":{"date-parts":[[2023,12,16]],"date-time":"2023-12-16T07:01:42Z","timestamp":1702710102000},"page":"13465-13494","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":11,"title":["Unobtrusive measurement of self-regulated learning: A clickstream-based multi-dimensional scale"],"prefix":"10.1007","volume":"29","author":[{"given":"Tudor","family":"Cristea","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Chris","family":"Snijders","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Uwe","family":"Matzat","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ad","family":"Kleingeld","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2023,12,16]]},"reference":[{"issue":"4","key":"12372_CR1","doi-asserted-by":"publisher","first-page":"759","DOI":"10.1007\/s11257-019-09234-7","volume":"29","author":"D Azcona","year":"2019","unstructured":"Azcona, D., Hsiao, I.-H., & Smeaton, A. F. (2019). Detecting students-at-risk in computer programming classes with learning analytics from students\u2019 digital footprints. User Modeling and User-Adapted Interaction, 29(4), 759\u2013788. https:\/\/doi.org\/10.1007\/s11257-019-09234-7","journal-title":"User Modeling and User-Adapted Interaction"},{"key":"12372_CR2","unstructured":"Beer, C., Clark, K., & Jones, D. (2010). Indicators of engagement. Curriculum, Technology & Transformation for an Unknown Future. Proceedings Ascilite Sydney 2010, 75\u201386. http:\/\/ascilite.org.au\/conferences\/sydney10\/procs\/Beer-full.pdf"},{"key":"12372_CR3","doi-asserted-by":"publisher","first-page":"928632","DOI":"10.3389\/feduc.2022.928632","volume":"7","author":"EB Cloude","year":"2022","unstructured":"Cloude, E. B., Azevedo, R., Winne, P. H., Biswas, G., & Jang, E. E. (2022). System design for using multimodal trace data in modeling self-regulated learning. Frontiers in Education, 7, 928632. https:\/\/doi.org\/10.3389\/feduc.2022.928632","journal-title":"Frontiers in Education"},{"issue":"1","key":"12372_CR4","doi-asserted-by":"publisher","first-page":"155","DOI":"10.1037\/0033-2909.112.1.155","volume":"112","author":"J Cohen","year":"1992","unstructured":"Cohen, J. (1992). A power primer. Psychological Bulletin, 112(1), 155\u2013159.","journal-title":"Psychological Bulletin"},{"issue":"1","key":"12372_CR5","doi-asserted-by":"publisher","first-page":"17","DOI":"10.1109\/TLT.2016.2616312","volume":"10","author":"R Conijn","year":"2017","unstructured":"Conijn, R., Snijders, C., Kleingeld, A., & Matzat, U. (2017). Predicting student performance from LMS Data: A comparison of 17 blended courses using moodle LMS. IEEE Transactions on Learning Technologies, 10(1), 17\u201329. https:\/\/doi.org\/10.1109\/TLT.2016.2616312","journal-title":"IEEE Transactions on Learning Technologies"},{"issue":"2","key":"12372_CR6","doi-asserted-by":"publisher","first-page":"129","DOI":"10.1007\/s11251-007-9039-0","volume":"37","author":"R del Valle","year":"2009","unstructured":"del Valle, R., & Duffy, T. M. (2009). Online learning: Learner characteristics and their approaches to managing learning. Instructional Science, 37(2), 129\u2013149. https:\/\/doi.org\/10.1007\/s11251-007-9039-0","journal-title":"Instructional Science"},{"issue":"1","key":"12372_CR7","doi-asserted-by":"publisher","first-page":"59","DOI":"10.1109\/TLT.2018.2823317","volume":"12","author":"E Fincham","year":"2019","unstructured":"Fincham, E., Gasevic, D., Jovanovi\u0107, J., & Pardo, A. (2019). From study tactics to learning strategies: An analytical method for extracting interpretable representations. IEEE Transactions on Learning Technologies, 12(1), 59\u201372. https:\/\/doi.org\/10.1109\/TLT.2018.2823317","journal-title":"IEEE Transactions on Learning Technologies"},{"key":"12372_CR8","doi-asserted-by":"publisher","first-page":"68","DOI":"10.1016\/j.iheduc.2015.10.002","volume":"28","author":"D Ga\u0161evi\u0107","year":"2016","unstructured":"Ga\u0161evi\u0107, D., Dawson, S., Rogers, T., & Ga\u0161evi\u0107, D. (2016). Learning analytics should not promote one size fits all: The effects of instructional conditions in predicting academic success. The Internet and Higher Education, 28, 68\u201384. https:\/\/doi.org\/10.1016\/j.iheduc.2015.10.002","journal-title":"The Internet and Higher Education"},{"key":"12372_CR9","doi-asserted-by":"publisher","first-page":"107304","DOI":"10.1016\/j.chb.2022.107304","volume":"134","author":"D Gasevic","year":"2022","unstructured":"Ga\u0161evi\u0107, D., Greiff, S., & Shaffer, D. W. (2022). Towards strengthening links between learning analytics and assessment: Challenges and potentials of a promising new bond. Computers in Human Behavior, 134, 107304. https:\/\/doi.org\/10.1016\/j.chb.2022.107304","journal-title":"Computers in Human Behavior"},{"issue":"1","key":"12372_CR10","doi-asserted-by":"publisher","first-page":"18","DOI":"10.1016\/j.cedpsych.2008.05.006","volume":"34","author":"JA Greene","year":"2009","unstructured":"Greene, J. A., & Azevedo, R. (2009). A macro-level analysis of SRL processes and their relations to the acquisition of a sophisticated mental model of a complex system. Contemporary Educational Psychology, 34(1), 18\u201329. https:\/\/doi.org\/10.1016\/j.cedpsych.2008.05.006","journal-title":"Contemporary Educational Psychology"},{"key":"12372_CR11","doi-asserted-by":"publisher","first-page":"104496","DOI":"10.1016\/j.compedu.2022.104496","volume":"183","author":"J Hutain","year":"2022","unstructured":"Hutain, J., & Michinov, N. (2022). Improving student engagement during in-person classes by using functionalities of a digital learning environment. Computers & Education, 183, 104496. https:\/\/doi.org\/10.1016\/j.compedu.2022.104496","journal-title":"Computers & Education"},{"key":"12372_CR12","doi-asserted-by":"publisher","first-page":"74","DOI":"10.1016\/j.iheduc.2017.02.001","volume":"33","author":"J Jovanovi\u0107","year":"2017","unstructured":"Jovanovi\u0107, J., Ga\u0161evi\u0107, D., Dawson, S., Pardo, A., & Mirriahi, N. (2017). Learning analytics to unveil learning strategies in a flipped classroom. The Internet and Higher Education, 33, 74\u201385. https:\/\/doi.org\/10.1016\/j.iheduc.2017.02.001","journal-title":"The Internet and Higher Education"},{"key":"12372_CR13","doi-asserted-by":"publisher","unstructured":"Jovanovi\u0107, J., Ga\u0161evi\u0107, D., Pardo, A., Dawson, S., & Whitelock-Wainwright, A. (2019). Introducing meaning to clicks: Towards traced-measures of self-efficacy and cognitive load. Proceedings of the 9th International Conference on Learning Analytics & Knowledge, 511\u2013520. https:\/\/doi.org\/10.1145\/3303772.3303782","DOI":"10.1145\/3303772.3303782"},{"key":"12372_CR14","doi-asserted-by":"publisher","first-page":"111","DOI":"10.1177\/001316447403400115","volume":"34","author":"HF Kaiser","year":"1974","unstructured":"Kaiser, H. F., & Rice, J. (1974). Little Jiffy, Mark Iv. Educational and Psychological Measurement, 34, 111\u2013117. https:\/\/doi.org\/10.1177\/001316447403400115","journal-title":"Educational and Psychological Measurement"},{"key":"12372_CR15","doi-asserted-by":"publisher","DOI":"10.1007\/s12528-022-09340-3","author":"M Khalil","year":"2022","unstructured":"Khalil, M., Prinsloo, P., & Slade, S. (2022). The use and application of learning theory in learning analytics: A scoping review. Journal of Computing in Higher Education. https:\/\/doi.org\/10.1007\/s12528-022-09340-3","journal-title":"Journal of Computing in Higher Education"},{"key":"12372_CR16","doi-asserted-by":"publisher","unstructured":"Kuhn, M. (2008). Building predictive models in R using the caret package. Journal of Statistical Software, 28(5). https:\/\/doi.org\/10.18637\/jss.v028.i05","DOI":"10.18637\/jss.v028.i05"},{"key":"12372_CR17","doi-asserted-by":"publisher","first-page":"286","DOI":"10.1016\/j.compedu.2017.07.007","volume":"114","author":"L-Y Li","year":"2017","unstructured":"Li, L.-Y., & Tsai, C.-C. (2017). Accessing online learning material: Quantitative behavior patterns and their effects on motivation and learning performance. Computers & Education, 114, 286\u2013297. https:\/\/doi.org\/10.1016\/j.compedu.2017.07.007","journal-title":"Computers & Education"},{"key":"12372_CR18","unstructured":"Long, P., & Siemens, G. (2011). Penetrating the fog: Analytics in learning and education. EDUCAUSE Review, 46(5), 31\u201340. Retrieved from https:\/\/er.educause.edu\/-\/media\/files\/article-downloads\/erm1151.pdf"},{"key":"12372_CR19","doi-asserted-by":"crossref","unstructured":"Maier, M. J. (2015). Companion package to the book \u201cR: Einf\u00fchrung durch angewandte Statistik\u201d (0.9.4) [Computer software]. http:\/\/CRAN.R-project.org\/package=REdaS","DOI":"10.32614\/CRAN.package.REdaS"},{"issue":"2","key":"12372_CR20","doi-asserted-by":"publisher","first-page":"i","DOI":"10.1002\/j.2330-8516.1987.tb00244.x","volume":"1987","author":"S Messick","year":"1987","unstructured":"Messick, S. (1987). VALIDITY. ETS Research Report Series, 1987(2), i\u2013208. https:\/\/doi.org\/10.1002\/j.2330-8516.1987.tb00244.x","journal-title":"ETS Research Report Series"},{"key":"12372_CR21","doi-asserted-by":"publisher","unstructured":"Milligan, S. K. (2018). Methodological foundations for the measurement of learning in learning analytics. Proceedings of the 8th International Conference on Learning Analytics and Knowledge, 466\u2013470. https:\/\/doi.org\/10.1145\/3170358.3170391","DOI":"10.1145\/3170358.3170391"},{"key":"12372_CR22","unstructured":"Nunnally, J. C. (1978). Psychometric theory (2nd ed.). McGraw-Hill."},{"key":"12372_CR23","doi-asserted-by":"publisher","first-page":"422","DOI":"10.3389\/fpsyg.2017.00422","volume":"8","author":"E Panadero","year":"2017","unstructured":"Panadero, E. (2017). A review of self-regulated learning: Six models and four directions for research. Frontiers in Psychology, 8, 422. https:\/\/doi.org\/10.3389\/fpsyg.2017.00422","journal-title":"Frontiers in Psychology"},{"issue":"6","key":"12372_CR24","doi-asserted-by":"publisher","first-page":"723","DOI":"10.1080\/00313831.2015.1066436","volume":"60","author":"E Panadero","year":"2016","unstructured":"Panadero, E., Klug, J., & J\u00e4rvel\u00e4, S. (2016). Third wave of measurement in the self-regulated learning field: When measurement and intervention come hand in hand. Scandinavian Journal of Educational Research, 60(6), 723\u2013735. https:\/\/doi.org\/10.1080\/00313831.2015.1066436","journal-title":"Scandinavian Journal of Educational Research"},{"key":"12372_CR25","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1016\/j.iheduc.2015.11.001","volume":"29","author":"Y Park","year":"2016","unstructured":"Park, Y., Yu, J. H., & Jo, I.-H. (2016). Clustering blended learning courses by online behavior data: A case study in a Korean higher education institute. The Internet and Higher Education, 29, 1\u201311. https:\/\/doi.org\/10.1016\/j.iheduc.2015.11.001","journal-title":"The Internet and Higher Education"},{"key":"12372_CR26","doi-asserted-by":"publisher","unstructured":"Pintrich, P., Duncan, T., Smith, D., & Mckeachie, W. (1991). Motivated strategies for learning questionnaire (MSLQ) manual. https:\/\/doi.org\/10.13140\/RG.2.1.2547.6968","DOI":"10.13140\/RG.2.1.2547.6968"},{"key":"12372_CR27","doi-asserted-by":"publisher","unstructured":"Quick, J., Motz, B., Israel, J., & Kaetzel, J. (2020). What college students say, and what they do: Aligning self-regulated learning theory with behavioral logs. Proceedings of the Tenth International Conference on Learning Analytics & Knowledge, 534\u2013543. https:\/\/doi.org\/10.1145\/3375462.3375516","DOI":"10.1145\/3375462.3375516"},{"key":"12372_CR28","unstructured":"R Core Team. (2022). R: A language and environment for statistical computing (4.2.2) [Computer software]. https:\/\/www.R-project.org\/"},{"key":"12372_CR29","unstructured":"Revelle, W. (2022). psych: Procedures for Personality and Psychological Research (2.2.3) [Computer software]. Northwestern University. https:\/\/CRAN.R-project.org\/package=psych"},{"key":"12372_CR30","doi-asserted-by":"publisher","first-page":"104325","DOI":"10.1016\/j.compedu.2021.104325","volume":"175","author":"M Saqr","year":"2021","unstructured":"Saqr, M., & L\u00f3pez-Pernas, S. (2021). The longitudinal trajectories of online engagement over a full program. Computers & Education, 175, 104325. https:\/\/doi.org\/10.1016\/j.compedu.2021.104325","journal-title":"Computers & Education"},{"issue":"12","key":"12372_CR31","doi-asserted-by":"publisher","first-page":"2370","DOI":"10.1080\/03075079.2022.2061450","volume":"47","author":"M Saqr","year":"2022","unstructured":"Saqr, M., Jovanovic, J., Viberg, O., & Ga\u0161evi\u0107, D. (2022). Is there order in the mess? A single paper meta-analysis approach to identification of predictors of success in learning analytics. Studies in Higher Education, 47(12), 2370\u20132391. https:\/\/doi.org\/10.1080\/03075079.2022.2061450","journal-title":"Studies in Higher Education"},{"key":"12372_CR32","doi-asserted-by":"publisher","unstructured":"Siadaty, M., Ga\u0161evi\u0107, D., & Hatala, M. (2016). Trace-based micro-analytic measurement of self-regulated learning processes. Journal of Learning Analytics, 3(1). https:\/\/doi.org\/10.18608\/jla.2016.31.11","DOI":"10.18608\/jla.2016.31.11"},{"issue":"6","key":"12372_CR33","doi-asserted-by":"publisher","first-page":"1273","DOI":"10.1007\/s11165-016-9602-2","volume":"48","author":"KS Taber","year":"2018","unstructured":"Taber, K. S. (2018). The use of Cronbach\u2019s Alpha when developing and reporting research instruments in science education. Research in Science Education, 48(6), 1273\u20131296. https:\/\/doi.org\/10.1007\/s11165-016-9602-2","journal-title":"Research in Science Education"},{"issue":"1","key":"12372_CR34","doi-asserted-by":"publisher","first-page":"70","DOI":"10.1111\/jcal.12392","volume":"36","author":"NA Uzir","year":"2020","unstructured":"Uzir, N. A., Ga\u0161evi\u0107, D., Matcha, W., Jovanovi\u0107, J., & Pardo, A. (2020b). Analytics of time management strategies in a flipped classroom. Journal of Computer Assisted Learning, 36(1), 70\u201388. https:\/\/doi.org\/10.1111\/jcal.12392","journal-title":"Journal of Computer Assisted Learning"},{"key":"12372_CR35","doi-asserted-by":"publisher","unstructured":"Uzir, N. A., Ga\u0161evi\u0107, D., Jovanovi\u0107, J., Matcha, W., Lim, L.-A., & Fudge, A. (2020). Analytics of time management and learning strategies for effective online learning in blended environments. Proceedings of the Tenth International Conference on Learning Analytics & Knowledge, 392\u2013401. https:\/\/doi.org\/10.1145\/3375462.3375493","DOI":"10.1145\/3375462.3375493"},{"key":"12372_CR36","doi-asserted-by":"publisher","unstructured":"Viberg, O., Khalil, M., & Baars, M. (2020). Self-regulated learning and learning analytics in online learning environments: A review of empirical research. Proceedings of the Tenth International Conference on Learning Analytics & Knowledge, 524\u2013533. https:\/\/doi.org\/10.1145\/3375462.3375483","DOI":"10.1145\/3375462.3375483"},{"issue":"4","key":"12372_CR37","doi-asserted-by":"publisher","first-page":"267","DOI":"10.1080\/00461520.2010.517150","volume":"45","author":"PH Winne","year":"2010","unstructured":"Winne, P. H. (2010). Improving measurements of self-regulated learning. Educational Psychologist, 45(4), 267\u2013276. https:\/\/doi.org\/10.1080\/00461520.2010.517150","journal-title":"Educational Psychologist"},{"issue":"2","key":"12372_CR38","doi-asserted-by":"publisher","first-page":"229","DOI":"10.1007\/s11409-014-9113-3","volume":"9","author":"PH Winne","year":"2014","unstructured":"Winne, P. H. (2014). Issues in researching self-regulated learning as patterns of events. Metacognition and Learning, 9(2), 229\u2013237. https:\/\/doi.org\/10.1007\/s11409-014-9113-3","journal-title":"Metacognition and Learning"},{"key":"12372_CR39","unstructured":"Winne, P. H., & Hadwin, A. F. (1998). Studying as self-regulated engagement in learning. In D. Hacker, J. Dunlosky, & A. Graesser (Eds.), Metacognition in educational theory and practice (pp. 277\u2013304). Hillsdale, NJ: Erlbaum."},{"key":"12372_CR40","doi-asserted-by":"publisher","first-page":"23","DOI":"10.1016\/j.iheduc.2015.11.003","volume":"29","author":"JW You","year":"2016","unstructured":"You, J. W. (2016). Identifying significant indicators using LMS data to predict course achievement in online learning. The Internet and Higher Education, 29, 23\u201330. https:\/\/doi.org\/10.1016\/j.iheduc.2015.11.003","journal-title":"The Internet and Higher Education"},{"key":"12372_CR41","doi-asserted-by":"publisher","first-page":"44","DOI":"10.1016\/j.iheduc.2015.05.002","volume":"27","author":"NZ Zacharis","year":"2015","unstructured":"Zacharis, N. Z. (2015). A multivariate approach to predicting student outcomes in web-enabled blended learning courses. The Internet and Higher Education, 27, 44\u201353. https:\/\/doi.org\/10.1016\/j.iheduc.2015.05.002","journal-title":"The Internet and Higher Education"}],"container-title":["Education and Information Technologies"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10639-023-12372-6.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10639-023-12372-6\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10639-023-12372-6.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,8,28]],"date-time":"2024-08-28T07:14:54Z","timestamp":1724829294000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10639-023-12372-6"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,12,16]]},"references-count":41,"journal-issue":{"issue":"11","published-print":{"date-parts":[[2024,8]]}},"alternative-id":["12372"],"URL":"https:\/\/doi.org\/10.1007\/s10639-023-12372-6","relation":{},"ISSN":["1360-2357","1573-7608"],"issn-type":[{"value":"1360-2357","type":"print"},{"value":"1573-7608","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,12,16]]},"assertion":[{"value":"29 May 2023","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"20 November 2023","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"16 December 2023","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors declare that they have no competing interests.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}]}}