{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,11,18]],"date-time":"2025-11-18T09:30:24Z","timestamp":1763458224124,"version":"build-2065373602"},"reference-count":53,"publisher":"MDPI AG","issue":"3","license":[{"start":{"date-parts":[[2023,2,22]],"date-time":"2023-02-22T00:00:00Z","timestamp":1677024000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001659","name":"German Research Foundation (DFG)","doi-asserted-by":"publisher","award":["AT 88\/4-1"],"award-info":[{"award-number":["AT 88\/4-1"]}],"id":[{"id":"10.13039\/501100001659","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Data"],"abstract":"<jats:p>There has recently been an increasing interest in Learning Management Systems (LMSs). It is currently unclear, however, exactly how these systems are perceived by their users. This article analyzes data on user acceptance for two LMSs (Blackboard and Canvas). The respective data are collected using a questionnaire modeled after the Technology Acceptance Model (TAM); it relates several variables that influence system acceptability, allowing for a detailed analysis of the system acceptance. We present analyses at two levels of the questionnaire data: questions and constructs (taken from TAM) as well as on different analysis levels using targeted methods. First, we investigate the differences between the above LMSs using statistical tests (t-test). Second, we provide results at the question level using descriptive indices, such as the mean and the Gini heterogeneity index, and apply methods for ordinal data using the Cumulative Link Mixed Model (CLMM). Next, we apply the same approach at the TAM construct level plus descriptive network analysis (degree centrality and bipartite motifs) to explore the variability of users\u2019 answers and the degree of users\u2019 satisfaction considering the extracted patterns. In the context of TAM, the statistical model is able to analyze LMS acceptance on the question level. As we are also very much interested in identifying LMS acceptance at the construct level, in this article, we provide both statistical analysis as well as network analysis to explore the connection between questionnaire data and relational data. A network analysis approach is particularly useful when analyzing LMS acceptance on the construct level, as this can take the structure of the users\u2019 answers across questions per construct into account. Taken together, these results suggest a higher rate of user acceptance among Canvas users compared to Blackboard both for the question and construct level. Likewise, the descriptive network modeling for Canvas indicates a slightly higher concordance between Canvas users than Blackboard at the construct level.<\/jats:p>","DOI":"10.3390\/data8030045","type":"journal-article","created":{"date-parts":[[2023,2,23]],"date-time":"2023-02-23T02:57:34Z","timestamp":1677121054000},"page":"45","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":10,"title":["Multi-Level Analysis of Learning Management Systems\u2019 User Acceptance Exemplified in Two System Case Studies"],"prefix":"10.3390","volume":"8","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-2193-2232","authenticated-orcid":false,"given":"Parisa","family":"Shayan","sequence":"first","affiliation":[{"name":"School of Humanities and Digital Sciences, Cognitive Science and Artificial Intelligence, Tilburg University, 5037 AB Tilburg, The Netherlands"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8510-8162","authenticated-orcid":false,"given":"Roberto","family":"Rondinelli","sequence":"additional","affiliation":[{"name":"Department of Economics and Statistics, University of Naples Federico II, Via Cintia 26, 80126 Naples, Italy"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1841-2444","authenticated-orcid":false,"given":"Menno","family":"van Zaanen","sequence":"additional","affiliation":[{"name":"South African Centre for Digital Language Resources, North-West University, Potchefstroom 2520, South Africa"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2480-6901","authenticated-orcid":false,"given":"Martin","family":"Atzmueller","sequence":"additional","affiliation":[{"name":"Semantic Information Systems Group, Osnabr\u00fcck University, 49090 Osnabr\u00fcck, Germany"},{"name":"German Research Center for Artificial Intelligence (DKFI), 49090 Osnabr\u00fcck, Germany"}]}],"member":"1968","published-online":{"date-parts":[[2023,2,22]]},"reference":[{"key":"ref_1","unstructured":"Szabo, M., and Flesher, K. (2002, January 15\u201319). CMI Theory and Practice: Historical Roots of Learning Management Systems. Proceedings of the E-Learn: World Conference on E-Learning in Corporate, Government, Healthcare, and Higher Education, Montreal, QC, Canada."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Shayan, P., Rondinelli, R., Zaanen, M., and Atzmueller, M. (2019, January 17). Descriptive Network Modeling and Analysis for Investigating User Acceptance in a Learning Management System Context. Proceedings of the ABIS \u201919: 23rd International Workshop on Personalization and Recommendation on the Web and Beyond, Hof, Germany.","DOI":"10.1145\/3345002.3349288"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"319","DOI":"10.2307\/249008","article-title":"Perceived usefulness, perceived ease of use, and user acceptance of information technology","volume":"13","author":"Davis","year":"1989","journal-title":"MIS Q."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"244","DOI":"10.2307\/2065853","article-title":"Belief, Attitude, Intention and Behavior: An Introduction to Theory and Research","volume":"6","author":"Hill","year":"1977","journal-title":"Contemp. Sociol."},{"key":"ref_5","unstructured":"Milo\u0161evi\u0107, M., Ze\u0107irovi\u0107, E., and Krneta, R. (2014, January 22\u201323). Technology Acceptance Models And Learning Management Systems: Case Study. Proceedings of the Fifth International Conference on e-Learning, Belgrade, Serbia."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"186","DOI":"10.1287\/mnsc.46.2.186.11926","article-title":"A Theoretical Extension of the Technology Acceptance Model: Four Longitudinal Field Studies","volume":"46","author":"Venkatesh","year":"2000","journal-title":"Manag. Sci."},{"key":"ref_7","unstructured":"Karrer, T. (2006, January 10\u201313). Understanding E-Learning 2.0: Introduction to Tools and Technologies. Proceedings of the Elearning Development Conference and Expo, San Francisco, CA, USA."},{"key":"ref_8","unstructured":"Caballero, J.A., Palomo, M., Dodero, J.M., Rodr\u00edguez, G., and Ibarra, M.S. (2014, January 22\u201323). Integrating External Evidence of Skill Assessment In Virtual Learning Environments. Proceedings of the Fifth International Conference on e-Learning, Belgrade, Serbia."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"301","DOI":"10.2190\/X137-X73L-5261-5656","article-title":"The Blackboard learning system: The be all and end all in educational instruction?","volume":"35","author":"Bradford","year":"2007","journal-title":"J. Educ. Technol. Syst."},{"key":"ref_10","unstructured":"Baepler, P. (2017). The Canvas Learning Management System: Instructor And Student Experience, Center for Educational Innovation (CEI), University of Minnesota. Technical Report."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"315","DOI":"10.1080\/03050060903184916","article-title":"The transfer, translation, and transformation of educational processes: And their shape-shifting","volume":"45","author":"Cowen","year":"2009","journal-title":"Comp. Educ."},{"key":"ref_12","first-page":"175","article-title":"Technology Acceptance Model: A Survey of Literature","volume":"2","author":"Surendran","year":"2012","journal-title":"Int. J. Bus. Soc. Res. (IJBSR)"},{"key":"ref_13","first-page":"20","article-title":"Factors influencing faculty members\u2019 Learning Management Systems adoption behavior: An analysis using the Technology Acceptance Model","volume":"2","author":"Fathema","year":"2013","journal-title":"Int. J. Trends Econ. Manag. Technol. (IJTEMT)"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"42","DOI":"10.4018\/ijwp.2014040104","article-title":"Student acceptance of university web portals: A quantitative study","volume":"6","author":"Fathema","year":"2014","journal-title":"Int. J. Web Portals"},{"key":"ref_15","first-page":"210","article-title":"Expanding The Technology Acceptance Model (TAM) to Examine Faculty Use of Learning Management Systems (LMSs) In Higher Education Institutions","volume":"11","author":"Fathema","year":"2015","journal-title":"J. Online Learn. Teach."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"425","DOI":"10.2307\/30036540","article-title":"User Acceptance of Information Technology: Toward a Unified View","volume":"27","author":"Venkatesh","year":"2003","journal-title":"MIS Q."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Stieninger, M., and Nedbal, D. (2014, January 6\u20139). Diffusion and Acceptance of Cloud Computing in SMEs: Towards a Valence Model of Relevant Factors. Proceedings of the 2014 47th Hawaii International Conference on System Sciences, Washington, DC, USA.","DOI":"10.1109\/HICSS.2014.410"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"185","DOI":"10.1080\/07421222.1996.11518128","article-title":"An Empirical Assessment of a Modified Technology Acceptance Model","volume":"13","author":"Chau","year":"1996","journal-title":"J. Manag. Inf. Syst."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"204","DOI":"10.1287\/isre.9.2.204","article-title":"A Conceptual and Operational Definition of Personal Innovativeness in the Domain of Information Technology","volume":"9","author":"Agarwal","year":"1998","journal-title":"Inf. Syst. Res."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"273","DOI":"10.1111\/j.1540-5915.2008.00192.x","article-title":"Technology Acceptance Model 3 and a Research Agenda on Interventions","volume":"39","author":"Venkatesh","year":"2008","journal-title":"Decis. Sci."},{"key":"ref_21","unstructured":"Van der Heijden, H. (2000). Using the Technology Acceptance Model to Predict Website Usage: Extensions and Empirical Test, University of Amsterdam. Technical Report, Serie Research Memoranda, Research Memorandum."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"217","DOI":"10.1016\/S0378-7206(00)00061-6","article-title":"Extending the TAM for a World-Wide-Web context","volume":"38","author":"Moon","year":"2001","journal-title":"Inf. Manag."},{"key":"ref_23","first-page":"41","article-title":"Online shopping acceptance model\u2014A critical survey of consumer factors in online shopping","volume":"8","author":"Zhou","year":"2007","journal-title":"J. Electron. Commer. Res."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"31","DOI":"10.1016\/j.jretconser.2008.08.002","article-title":"Customer acceptance of RFID technology: Evidence from the German electronic retail sector","volume":"16","author":"Dautzenberg","year":"2009","journal-title":"J. Retail. Consum. Serv."},{"key":"ref_25","first-page":"243","article-title":"Case study of application-based mobile service acceptance and development in Finland","volume":"9","author":"Ervasti","year":"2010","journal-title":"Int. J. Inf. Technol. Manag."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"699","DOI":"10.1111\/j.1540-5915.2001.tb00978.x","article-title":"Information Technology Acceptance by Individual Professionals: A Model of Comparison Approach","volume":"32","author":"Chau","year":"2001","journal-title":"Decis. Sci."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"33","DOI":"10.14257\/ijmue.2015.10.1.4","article-title":"What Affects Lower Grade Learner\u2019s Perceived Usefulness and Perceived Ease of Use of Mobile Digital Textbook Learning System? An Empirical Factor Analyses Investigation in China","volume":"10","author":"Shen","year":"2015","journal-title":"Int. J. Multimed. Ubiquitous Eng."},{"key":"ref_28","first-page":"4","article-title":"The usage of technology among education students in university Uttara Malaysia: An application of extended technology acceptance model","volume":"7","author":"Arumugam","year":"2011","journal-title":"Int. J. Educ. Dev. Using Inf. Commun. Technol. (IJEDICT)"},{"key":"ref_29","first-page":"409","article-title":"Student acceptance of learning management systems: A study on demographics","volume":"15","author":"Claar","year":"2014","journal-title":"Issues Inf. Syst."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"895","DOI":"10.1080\/09720073.2014.11891504","article-title":"The factors that affect online learners\u2019 satisfaction","volume":"17","author":"Cakir","year":"2014","journal-title":"Anthropologist"},{"key":"ref_31","unstructured":"Dahlstrom, E., Brooks, D.C., and Bichsel, J. (2014). The Current Ecosystem of Learning Management Systems in Higher Education: Student, Faculty, and Its Perspectives, EDUCAUSE Center for Analysis and Research. Technical Report."},{"key":"ref_32","unstructured":"Kurkinen, E. (2013, January 12\u201315). The effect of age on technology acceptance among field police officers. Proceedings of the 10th International ISCRAM Conference, Baden-Baden, Germany."},{"key":"ref_33","first-page":"108","article-title":"Assessing students\u2019 satisfaction on the use of virtual learning environment (VLE): An input to a campus-wide e-learning design and implementation","volume":"3","author":"Chua","year":"2014","journal-title":"Inf. Knowl. Manag."},{"key":"ref_34","first-page":"1","article-title":"Online student satisfaction: An examination of preference, asynchronous course elements and collaboration among online students","volume":"17","author":"Marmon","year":"2014","journal-title":"Curr. Issues Educ. (CIE)"},{"key":"ref_35","first-page":"13","article-title":"Using the Technology Acceptance Model in Understanding Academics\u2019 Behavioural Intention to Use Learning Management Systems","volume":"5","author":"Alharbi","year":"2014","journal-title":"Int. J. Adv. Comput. Sci. Appl."},{"key":"ref_36","first-page":"1","article-title":"Comparing the Usability and Users Acceptance of Open Sources Learning Management System (LMS)","volume":"5","author":"Hock","year":"2015","journal-title":"Int. J. Sci. Res. Publ."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"421","DOI":"10.1007\/s10888-011-9188-x","article-title":"The origins of the Gini index: Extracts from Variabilit\u00e0 e Mutabilit\u00e0 (1912) by Corrado Gini","volume":"10","author":"Ceriani","year":"2012","journal-title":"J. Econ. Inequal."},{"key":"ref_38","unstructured":"Field, A., Miles, J., and Field, Z. (2012). Discovering Statistics Using R, SAGE Publications Ltd."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"167","DOI":"10.1137\/S003614450342480","article-title":"The structure and function of complex networks","volume":"45","author":"Newman","year":"2003","journal-title":"SIAM Rev."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"243","DOI":"10.1016\/S0378-8733(96)00301-2","article-title":"Network analysis of 2-mode data","volume":"19","author":"Clemente","year":"1997","journal-title":"Soc. Netw."},{"key":"ref_41","unstructured":"Wellman, B., and Berkowitz, S.D. (1988). Social Structures: A Network Approach, Cambridge University Press."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1103\/PhysRevPhysEducRes.18.020113","article-title":"Network analysis approach to Likert-style surveys","volume":"18","author":"Dalka","year":"2022","journal-title":"Phys. Rev. Phys. Educ. Res."},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"695","DOI":"10.1111\/2041-210X.13149","article-title":"Bmotif: A package for motif analyses of bipartite networks","volume":"10","author":"Simmons","year":"2018","journal-title":"Methodol. Ecology Evol."},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"154","DOI":"10.1111\/oik.05670","article-title":"Motifs in bipartite ecological networks: Uncovering indirect interactions","volume":"128","author":"Simmons","year":"2019","journal-title":"Oikos"},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"824","DOI":"10.1126\/science.298.5594.824","article-title":"Network Motifs: Simple Building Blocks of Complex Networks","volume":"298","author":"Milo","year":"2002","journal-title":"Science"},{"key":"ref_46","doi-asserted-by":"crossref","unstructured":"Poisot, T., and Stouffer, D. (2016). How ecological networks evolve. bioRxiv, bioRxiv: 071993.","DOI":"10.1101\/071993"},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"1266","DOI":"10.1111\/1365-2745.13942","article-title":"Drivers of individual-based, antagonistic interaction networks during plant range expansion","volume":"110","author":"Isla","year":"2022","journal-title":"J. Ecol."},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"130","DOI":"10.1111\/ecog.00913","article-title":"Species\u2019 roles in food webs show fidelity across a highly variable oak forest","volume":"38","author":"Baker","year":"2015","journal-title":"Ecography"},{"key":"ref_49","first-page":"1","article-title":"Students\u2019 perspective of learning management system: An empirical evidence of technology acceptance model in emerging countries","volume":"6","author":"Siang","year":"2015","journal-title":"Res. World"},{"key":"ref_50","doi-asserted-by":"crossref","first-page":"297","DOI":"10.1007\/BF02310555","article-title":"Coefficient alpha and the internal structure of tests","volume":"16","author":"Cronbach","year":"1951","journal-title":"Psychometrika"},{"key":"ref_51","unstructured":"Gini, C. (1912). Variabilit\u00e0 e Mutabilit\u00e0: Contributo allo Studio delle Distribuzioni e delle Relazioni Statistiche, Tipogr. di P. Cuppini. Studi Economici-Giuridici della Regia Facolt\u00e0 di Giurisprudenza, Anno III, Parte II."},{"key":"ref_52","unstructured":"Bruin, J. (2006). Newtest: Command to Compute New Test @ONLINE, UCLA: Statistical Consulting Group. Technical Report."},{"key":"ref_53","unstructured":"Christensen, R. (2018). Proceedings of the Cumulative Link Models, Technical University of Denmark."}],"container-title":["Data"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2306-5729\/8\/3\/45\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T18:39:37Z","timestamp":1760121577000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2306-5729\/8\/3\/45"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,2,22]]},"references-count":53,"journal-issue":{"issue":"3","published-online":{"date-parts":[[2023,3]]}},"alternative-id":["data8030045"],"URL":"https:\/\/doi.org\/10.3390\/data8030045","relation":{},"ISSN":["2306-5729"],"issn-type":[{"type":"electronic","value":"2306-5729"}],"subject":[],"published":{"date-parts":[[2023,2,22]]}}}