{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,12]],"date-time":"2025-10-12T04:13:14Z","timestamp":1760242394503,"version":"build-2065373602"},"reference-count":36,"publisher":"MDPI AG","issue":"3","license":[{"start":{"date-parts":[[2017,6,26]],"date-time":"2017-06-26T00:00:00Z","timestamp":1498435200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Computers"],"abstract":"<jats:p>Three-dimensional (3D) virtual environments bring people together in real time irrespective of their geographical location to facilitate collaborative learning and working together in an engaging and fulfilling way. However, it can be difficult to amass suitable data to gauge how well students perform in these environments. With this in mind, the current study proposes a methodology for monitoring students\u2019 learning experiences in 3D virtual worlds (VWs). It integrates a computer-based mechanism that mixes software agents with natural agents (users) in conjunction with a fuzzy logic model to reveal evidence of learning in collaborative pursuits to replicate the sort of observation that would normally be made in a conventional classroom setting. Software agents are used to infer the extent of interaction based on the number of clicks, the actions of users, and other events. Meanwhile, natural agents are employed in order to evaluate the students and the way in which they perform. This is beneficial because such an approach offers an effective method for assessing learning activities in 3D virtual environments.<\/jats:p>","DOI":"10.3390\/computers6030022","type":"journal-article","created":{"date-parts":[[2017,6,27]],"date-time":"2017-06-27T02:58:05Z","timestamp":1498532285000},"page":"22","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":1,"title":["Towards Recognising Learning Evidence in Collaborative Virtual Environments: A Mixed Agents Approach"],"prefix":"10.3390","volume":"6","author":[{"given":"Samah","family":"Felemban","sequence":"first","affiliation":[{"name":"Department of Computer Science and Electronic Engineering, University of Essex, Colchester CO4 3SQ, UK"},{"name":"Umm Alqura University, Makkah 24382, Saudi Arabia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Michael","family":"Gardner","sequence":"additional","affiliation":[{"name":"Department of Computer Science and Electronic Engineering, University of Essex, Colchester CO4 3SQ, UK"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Victor","family":"Callaghan","sequence":"additional","affiliation":[{"name":"Department of Computer Science and Electronic Engineering, University of Essex, Colchester CO4 3SQ, UK"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2017,6,26]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"603","DOI":"10.1016\/j.compedu.2010.02.021","article-title":"Fostering collaborative learning in Second Life: Metaphors and affordances","volume":"55","author":"Andreas","year":"2010","journal-title":"Comput. Educ."},{"key":"ref_2","unstructured":"Belazoui, A., Kazar, O., and Bourekkache, S. (2016). A Cooperative Multi-Agent System for modeling of authoring system in e-learning. J. E-Learn. Knowl. Soc., 12."},{"key":"ref_3","unstructured":"Felemban, S. (2015, January 13\u201314). Distributed pedagogical virtual machine (d-pvm). Proceedings of the Immersive Learning Research Network Conference (iLRN 2015), Prague, Czech."},{"key":"ref_4","unstructured":"Schouten, A.P., van den Hooff, B., and Feldberg, F. (2010, January 12\u201315). Real Decisions in Virtual Worlds: Team Collaboration and Decision Making in 3D Virtual Worlds. Proceedings of the International Conference on Information Systems (ICIS 2010), Saint Louis, MO, USA."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"10","DOI":"10.1111\/j.1467-8535.2009.01038.x","article-title":"What are the learning affordances of 3-D virtual environments?","volume":"41","author":"Dalgarno","year":"2010","journal-title":"Br. J. Educ. Technol."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"949","DOI":"10.1111\/j.1467-8535.2011.01263.x","article-title":"A taxonomy of virtual worlds usage in education","volume":"43","author":"Duncan","year":"2012","journal-title":"Br. J. Educ. Technol."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"59","DOI":"10.1016\/j.iheduc.2015.10.001","article-title":"The effects of sentiments and co-regulation on group performance in computer supported collaborative learning","volume":"28","author":"Zheng","year":"2016","journal-title":"Internet High. Educ."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"107","DOI":"10.1016\/j.ins.2017.04.002","article-title":"Combining trust and skills evaluation to form e-Learning classes in online social networks","volume":"405","author":"Messina","year":"2017","journal-title":"Inf. Sci."},{"key":"ref_9","first-page":"e6","article-title":"The Immersive Education Laboratory: Understanding affordances, structuring experiences, and creating constructivist, collaborative processes, in mixed-reality smart environments","volume":"14","author":"Gardner","year":"2014","journal-title":"EAI Endorsed Trans. Future Intell. Educ. Environ."},{"key":"ref_10","unstructured":"Weiss, G. (1999). Multiagent Systems: A Modern Approach to Distributed Artificial Intelligence, MIT Press."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"115","DOI":"10.1017\/S0269888900008122","article-title":"Intelligent agents: Theory and practice","volume":"10","author":"Wooldridge","year":"1995","journal-title":"Knowl. Eng. Rev."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"793","DOI":"10.1016\/j.is.2006.06.003","article-title":"CILIOS: Connectionist inductive learning and inter-ontology similarities for recommending information agents","volume":"32","author":"Rosaci","year":"2007","journal-title":"Inf. Syst."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"341","DOI":"10.1080\/08839514.2011.559907","article-title":"EVA: An evolutionary approach to mutual monitoring of learning information agents","volume":"25","author":"Rosaci","year":"2011","journal-title":"Appl. Artif. Intell."},{"key":"ref_14","unstructured":"S\u00e1nchez, J.A. (1997). A Taxonomy of Agents. Rapport Technique, ICT-Universidad de las Am\u00e9ricas-Puebla."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"121","DOI":"10.1111\/j.1467-8640.2009.00343.x","article-title":"Efficient Personalization of E-Learning Activities Using a Multi-Device Decentralized Recommender System","volume":"26","author":"Rosaci","year":"2010","journal-title":"Comput. Intell."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"205","DOI":"10.1017\/S026988890000789X","article-title":"Software agents: An overview","volume":"11","author":"Nwana","year":"1996","journal-title":"Knowl. Eng. Rev."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"229","DOI":"10.1016\/B978-1-55860-092-8.50014-9","article-title":"Negotiating task decomposition and allocation using partial global planning","volume":"2","author":"Durfee","year":"1989","journal-title":"Distrib. Artif. Intell."},{"key":"ref_18","first-page":"676","article-title":"Modeling academic performance evaluation using soft computing techniques: A fuzzy logic approach","volume":"3","author":"Yadav","year":"2011","journal-title":"Int. J. Comput. Sci. Eng."},{"key":"ref_19","unstructured":"Albertos, P., and Sala, A. (2002). Fuzzy Expert Control Systems: Knowledge Base Validation, UNESCO Encyclopedia of Life Support Systems."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Thompson, K., and Markauskaite, L. (2014). Identifying Group Processes and Affect in Learners: A Holistic Approach to. Cases on the Assessment of Scenario and Game-Based Virtual Worlds in Higher Education, IGI Global.","DOI":"10.4018\/978-1-4666-4470-0.ch006"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Alrashidi, M., Almohammadi, K., Gardner, M., and Callaghan, V. (2017, January 12\u201315). Making the Invisible Visible: Real-Time Feedback for Embedded Computing Learning Activity Using Pedagogical Virtual Machine with Augmented Reality. Proceedings of the International Conference on Augmented Reality, Virtual Reality and Computer Graphics, Ugento, Italy.","DOI":"10.1007\/978-3-319-60922-5_27"},{"key":"ref_22","first-page":"111","article-title":"Leveraging educational data mining for real-time performance assessment of scientific inquiry skills within microworlds","volume":"4","author":"Gobert","year":"2012","journal-title":"JEDM J. Educ. Data Min."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"337","DOI":"10.1207\/s15516709cog2303_3","article-title":"The generality\/specificity of expertise in scientific reasoning","volume":"23","author":"Schunn","year":"1999","journal-title":"Cogn. Sci."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Annetta, L.A., Folta, E., and Klesath, M. (2010). Assessing and Evaluating Virtual World Effectiveness. V-Learning, Springer.","DOI":"10.1007\/978-90-481-3627-8"},{"key":"ref_25","first-page":"144","article-title":"Identifying key features of student performance in educational video games and simulations through cluster analysis","volume":"4","author":"Kerr","year":"2012","journal-title":"JEDM J. Educ. Data Min."},{"key":"ref_26","first-page":"i-29","article-title":"A brief introduction to evidence-centered design","volume":"2003","author":"Mislevy","year":"2003","journal-title":"ETS Res. Rep. Ser."},{"key":"ref_27","unstructured":"Tesfazgi, S.H. (2003). Survey on Behavioral Observation Methods in Virtual Environments, Delft University of Technology. Research Assignment."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Griffin, P., McGaw, B., and Care, E. (2012). Technological Issues for Computer-Based Assessment. Assessment and Teaching of 21st Century Skills, Springer.","DOI":"10.1007\/978-94-007-2324-5"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Ib\u00e1\u00f1ez, M.B., Crespo, R.M., and Kloos, C.D. (2010). Assessment of knowledge and competencies in 3D virtual worlds: A proposal. Key Competencies in the Knowledge Society, Springer.","DOI":"10.1007\/978-3-642-15378-5_16"},{"key":"ref_30","first-page":"503","article-title":"Stealth assessment in computer-based games to support learning","volume":"55","author":"Shute","year":"2011","journal-title":"Comput. Games Instr."},{"key":"ref_31","unstructured":"Felemban, S., Gardner, M., and Challagan, V. (July, January 27). Virtual Observation Lenses for Assessing Online Collaborative Learning Environments. Proceedings of the Immersive Learning Research Network (iLRN 2016), Santa Barbra, CA, USA."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Beck, D., Allison, C., Morgado, L., Pirker, J., Khosmood, F., Richter, J., and G\u00fctl, C. (2017, January 26\u201329). Towards Observing and Assessing Collaborative Learning Activities in Immersive Environments. Proceedings of the Immersive Learning Research Network: Third International Conference (iLRN 2017), Coimbra, Portugal.","DOI":"10.1007\/978-3-319-60633-0"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Felemban, S., Gardner, M., and Callaghan, V. (2016, January 28\u201330). An event detection approach for identifying learning evidence in collaborative virtual environments. Proceedings of the Computer Science and Electronic Engineering (CEEC), Colchester, UK.","DOI":"10.1109\/CEEC.2016.7835886"},{"key":"ref_34","unstructured":"Excellence, E.C.F.T. (2017, May 31). How Can I Assess Group Work?. Available online: https:\/\/www.cmu.edu\/teaching\/designteach\/design\/instructionalstrategies\/groupprojects\/assess.html."},{"key":"ref_35","unstructured":"Padhy, N. (2005). Artificial Intelligence and Intelligent Systems, Oxford University Press."},{"key":"ref_36","unstructured":"Pena-Rios, A. (2016). Exploring Mixed Reality in Distributed Collaborative Learning Environments. School of Computer Science and Electronic Engineering, University of Essex."}],"container-title":["Computers"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2073-431X\/6\/3\/22\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T18:40:28Z","timestamp":1760208028000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2073-431X\/6\/3\/22"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2017,6,26]]},"references-count":36,"journal-issue":{"issue":"3","published-online":{"date-parts":[[2017,9]]}},"alternative-id":["computers6030022"],"URL":"https:\/\/doi.org\/10.3390\/computers6030022","relation":{},"ISSN":["2073-431X"],"issn-type":[{"type":"electronic","value":"2073-431X"}],"subject":[],"published":{"date-parts":[[2017,6,26]]}}}