{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,11,2]],"date-time":"2025-11-02T05:52:05Z","timestamp":1762062725110,"version":"build-2065373602"},"reference-count":65,"publisher":"MDPI AG","issue":"16","license":[{"start":{"date-parts":[[2022,8,21]],"date-time":"2022-08-21T00:00:00Z","timestamp":1661040000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Funda\u00e7\u00e3o para a Ci\u00eancia e Tecnologia","award":["UIDB\/00066\/2020","2020-1-FR01-KA202-080231 ED-EN HUB"],"award-info":[{"award-number":["UIDB\/00066\/2020","2020-1-FR01-KA202-080231 ED-EN HUB"]}]},{"name":"European Commission ERASMUS +","award":["UIDB\/00066\/2020","2020-1-FR01-KA202-080231 ED-EN HUB"],"award-info":[{"award-number":["UIDB\/00066\/2020","2020-1-FR01-KA202-080231 ED-EN HUB"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Applied Sciences"],"abstract":"<jats:p>Mass collaboration can bring about major transformative changes in the way people can work collectively. This emerging paradigm promises significant economic and social benefits and enhanced efficiency across a range of sectors, including learning and education. Accordingly, this article introduces, demonstrates in use, and evaluates an organizational and governance model designed to provide guidance and execution support for the implementation and operation of mass collaborative learning initiatives. The design science research process is adopted to guide the design and development of the proposed model. The model stands on three streams of work, addressing key aspects and elements that have a supporting influence on community learning: (i) identify the positive and negative factors in existing and active examples of mass collaboration; (ii) adopt contributions of collaborative networks in terms of structural and behavioral aspects; and (iii) establish adequate learning assessment indicators and metrics. The model is used for a case study in which vocational education and training meet the needs of collaborative education\u2013enterprise approaches. Initially, the validation of the model is verified by the partners and stakeholders of a particular project in the area of education\u2013enterprises relations to ensure that it is sufficiently appropriate for applications in a digital platform developed by such projects. The three first steps of (the proposed) applicability evaluation (adequacy, feasibility, and effectiveness) are then performed. The positive results gained from model validation and its applicability evaluation in this project indicate that not only is the model fairly adequate, feasible, and effective for applications in the developed digital platform but also that it has a high potential for utilization in supporting and directing the creation, implementation, and operation of mass collaborative learning initiatives. Although the validation was carried out in the context of a single project, in fact, it was based on a large \u201cfocus group\u201d of experts involved in this international initiative, which is in accordance with the Design Science Research method. Thus, this article reflects a kind of applied research of a socio-technical nature, aiming to find guidelines and practical solutions to the specific issues, problems, and concerns of mass collaborative learning initiatives.<\/jats:p>","DOI":"10.3390\/app12168356","type":"journal-article","created":{"date-parts":[[2022,8,21]],"date-time":"2022-08-21T21:05:51Z","timestamp":1661115951000},"page":"8356","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":3,"title":["An Organizational and Governance Model to Support Mass Collaborative Learning Initiatives"],"prefix":"10.3390","volume":"12","author":[{"given":"Majid","family":"Zamiri","sequence":"first","affiliation":[{"name":"School of Science and Technology and Center of Technology and Systems (CTS-Uninova), NOVA University of Lisbon, 2829-516 Monte de Caparica, Portugal"}]},{"given":"Jo\u00e3o","family":"Sarraipa","sequence":"additional","affiliation":[{"name":"School of Science and Technology and Center of Technology and Systems (CTS-Uninova), NOVA University of Lisbon, 2829-516 Monte de Caparica, Portugal"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0594-1961","authenticated-orcid":false,"given":"Luis","family":"Camarinha-Matos","sequence":"additional","affiliation":[{"name":"School of Science and Technology and Center of Technology and Systems (CTS-Uninova), NOVA University of Lisbon, 2829-516 Monte de Caparica, Portugal"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3703-6854","authenticated-orcid":false,"given":"Ricardo","family":"Jardim-Goncalves","sequence":"additional","affiliation":[{"name":"School of Science and Technology and Center of Technology and Systems (CTS-Uninova), NOVA University of Lisbon, 2829-516 Monte de Caparica, Portugal"}]}],"member":"1968","published-online":{"date-parts":[[2022,8,21]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Zamiri, M., and Camarinha-Matos, L.M. (2019). Mass Collaboration and Learning: Opportunities, Challenges, and Influential Factors. Appl. Sci., 9.","DOI":"10.3390\/app9132620"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"439","DOI":"10.1007\/s10845-005-1656-3","article-title":"Collaborative networks: A new scientific discipline","volume":"16","author":"Afsarmanesh","year":"2005","journal-title":"Intell. Manuf."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Cress, U., Moskaliuk, J., and Jeong, H. (2016). Mass Collaboration and Education, Springer.","DOI":"10.1007\/978-3-319-13536-6"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"14","DOI":"10.1007\/978-3-030-17771-3_2","article-title":"Organizational Structure for Mass Collaboration and Learning","volume":"Volume 553","author":"Zamiri","year":"2019","journal-title":"Technological Innovation for Industry and Service Systems"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"1569","DOI":"10.1007\/s11423-017-9535-0","article-title":"I don\u2019t think that word means what you think it means: A proposed framework for defining learning communities","volume":"65","author":"West","year":"2017","journal-title":"Educ. Technol. Res. Dev."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Zamiri, M., and Camarinha-Matos, L.M. (2021). Technological Innovation for Applied AI Systems, Springer. DoCEIS 2021. IFIP Advances in Information and Communication Technology.","DOI":"10.1007\/978-3-030-78288-7"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"433","DOI":"10.1080\/01449290802005995","article-title":"Determinants of success for online communities: An analysis of three communities in terms of members\u2019 perceived professional development","volume":"28","author":"Hew","year":"2009","journal-title":"Behav. Inf. Technol."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"709","DOI":"10.2307\/20650324","article-title":"Community Learning in Information Technology Innovation","volume":"33","author":"Wang","year":"2009","journal-title":"MIS Q."},{"key":"ref_9","unstructured":"Hipp, K.K., and Huffman, J.B. (2003, January 5\u20138). Professional Learning Communities: Assessment-Development-Effects. Proceedings of the International Congress for School Effectiveness and Improvement, Sydney, Australia."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"133","DOI":"10.1080\/21532974.2012.10784693","article-title":"Professional Learning Networks Designed for Teacher Learning","volume":"28","author":"Trust","year":"2012","journal-title":"J. Digit. Learn. Teach. Educ."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Wenger, E. (1999). Communities of Practice: Learning, Meaning, and Identity, Cambridge University Press. [1st ed.].","DOI":"10.1017\/CBO9780511803932"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Cress, U., Moskaliuk, J., and Jeong, H. (2016). Mass Collaboration and Education, Springer.","DOI":"10.1007\/978-3-319-13536-6"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Cress, U., Moskaliuk, J., and Jeong, H. (2016). Mass Collaboration and Education, Springer.","DOI":"10.1007\/978-3-319-13536-6"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Cress, U., Moskaliuk, J., and Jeong, H. (2016). Mass Collaboration and Education, Springer.","DOI":"10.1007\/978-3-319-13536-6"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Cress, U., Moskaliuk, J., and Jeong, H. (2016). Mass Collaboration and Education, Springer.","DOI":"10.1007\/978-3-319-13536-6"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Cress, U., Moskaliuk, J., and Jeong, H. (2016). Mass Collaboration and Education, Springer.","DOI":"10.1007\/978-3-319-13536-6"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Cress, U., Moskaliuk, J., and Jeong, H. (2016). Mass Collaboration and Education, Springer.","DOI":"10.1007\/978-3-319-13536-6"},{"key":"ref_18","unstructured":"Morris, T., and Mietchen, D. (2010, January 24). Collaborative Structuring of Knowledge by Experts and the Public. Proceedings of the 5th Open Knowledge Conference, London, UK."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"5","DOI":"10.1080\/01587919.2015.1019968","article-title":"MOOCs as a massive research laboratory: Opportunities and challenges","volume":"36","author":"Diver","year":"2015","journal-title":"Distance Educ."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"355","DOI":"10.1080\/1369118X.2010.542823","article-title":"From Encyclop\u00e6dia Britannica to Wikipedia\u2014Generational Differences in the Perceived Credibility of Online Encyclopedia Information","volume":"14","author":"Flanagin","year":"2011","journal-title":"Inf. Commun. Soc."},{"key":"ref_21","unstructured":"(2022, May 10). Wikipedia. Available online: https:\/\/en.wikipedia.org\/wiki\/Wikipedia."},{"key":"ref_22","unstructured":"Gunelius, S. (2022, May 10). Overview of Digg. Available online: https:\/\/www.thoughtco.com\/overview-of-digg-3476441."},{"key":"ref_23","unstructured":"(2022, May 10). Digg: What Is Digg?. Available online: https:\/\/www.wordstream.com\/digg."},{"key":"ref_24","unstructured":"(2022, May 10). Yahoo. Available online: https:\/\/www.yahoo.com\/."},{"key":"ref_25","unstructured":"(2022, May 10). SETI@home. Available online: https:\/\/setiathome.berkeley.edu\/."},{"key":"ref_26","unstructured":"(2022, May 10). Scratch. Available online: https:\/\/scratch.mit.edu\/."},{"key":"ref_27","unstructured":"(2022, May 10). GalaxyZoo. Available online: https:\/\/www.zooniverse.org\/projects\/zookeeper\/galaxy-zoo\/."},{"key":"ref_28","unstructured":"(2022, May 10). Foldit. Available online: https:\/\/fold.it\/."},{"key":"ref_29","unstructured":"(2022, May 10). Wikipedia. Available online: https:\/\/en.wikipedia.org\/wiki\/Delphi_method."},{"key":"ref_30","unstructured":"(2022, May 10). Climate CoLab. Available online: https:\/\/www.climatecolab.org\/page\/about."},{"key":"ref_31","unstructured":"(2022, May 10). Wikipedia. Available online: https:\/\/en.wikipedia.org\/wiki\/Assignment_Zero."},{"key":"ref_32","unstructured":"(2022, May 10). DonationCoder. Available online: https:\/\/www.donationcoder.com\/help\/faqs."},{"key":"ref_33","unstructured":"(2022, May 10). Expert Exchange. Available online: https:\/\/go.experts-exchange.com\/plansandpricing?mixedRegTrack=headerHome."},{"key":"ref_34","unstructured":"(2022, May 10). Waze. Available online: https:\/\/www.waze.com\/live-map\/."},{"key":"ref_35","unstructured":"(2022, May 10). Makerspace. Available online: https:\/\/www.makerspaces.com\/what-is-a-makerspace\/."},{"key":"ref_36","unstructured":"(2022, May 10). SAP Community. Available online: https:\/\/community.sap.com\/."},{"key":"ref_37","unstructured":"Camarinha-Matos, L.M., and Afsarmanesh, H. (2006, January 14\u201316). Collaborative Networks: Value creation in a knowledge society. Proceedings of the International Conference on Programming Languages for Manufacturing (PROLAMAT\u201906), Shanghai, China."},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Camarinha-Matos, L.M., Afsarmanesh, H., Ermilova, E., Ferrada, F., Klen, A., and Jarimo, T. (2008). Collaborative Network: Reference Modeling, Springer.","DOI":"10.1007\/978-0-387-79426-6"},{"key":"ref_39","unstructured":"Ellmann, S., and Eschenbaecher, J. (2005). Virtual Enterprise Integration: Technological and Organizational Perspectives, Idea Group Pub."},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Camarinha-Matos, L.M., and Afsarmanesh, H. (2011, January 26\u201329). Behavioral aspects in collaborative enterprise networks. Proceedings of the 9th IEEE International Conference on Industrial Informatics, Lisbon, Portugal.","DOI":"10.1109\/INDIN.2011.6034828"},{"key":"ref_41","first-page":"37","article-title":"Risk and Benefits Behavioral Model to Assess Intentions to Adopt Big Data","volume":"3","author":"Esteves","year":"2013","journal-title":"Intell. Stud. Bus."},{"key":"ref_42","first-page":"1120","article-title":"Towards an Education Behavioral Intention Model for E-Learning Systems: An Extension of UTAUT","volume":"47","author":"Li","year":"2013","journal-title":"Theor. Appl. Inf. Technol."},{"key":"ref_43","unstructured":"(2022, May 10). Leading SDG 4\u2014Education 2030. Available online: https:\/\/en.unesco.org\/themes\/education2030-sdg4."},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"8","DOI":"10.1016\/S0099-1333(01)00295-6","article-title":"Developing an assessment plan to learn about student learning","volume":"28","author":"Maki","year":"2002","journal-title":"Acad. Librariansh."},{"key":"ref_45","unstructured":"Harlen, W. (2007). Primary Review Research Survey 3\/4, Faculty of Education, University of Cambridge."},{"key":"ref_46","unstructured":"Nagowah, S.D., and Nagowah, L. (2009, January 7\u20139). Assessment Strategies to enhance Students\u2019 Success. Proceedings of the IASK International Conference \u201cTeaching and Learning\u201d, Porto, Portugal."},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"45","DOI":"10.52731\/iee.v2.i1.61","article-title":"Assessment Indicators as a Tool of Process Monitoring, Benchmarking and Learning Outcomes Assessment: Features of Two Types Indicators","volume":"2","author":"Aihara","year":"2016","journal-title":"Inf. Eng. Express"},{"key":"ref_48","first-page":"35","article-title":"Developing of Indicators of an E-Learning Benchmarking Model for Higher Education Institutions","volume":"13","year":"2014","journal-title":"Turk. Online J. Educ. Technol."},{"key":"ref_49","first-page":"117","article-title":"Quality Indicators for Learning Analytics","volume":"17","author":"Scheffel","year":"2014","journal-title":"Educ. Technol. Soc."},{"key":"ref_50","first-page":"1412","article-title":"Performance Management in Collaborative Networks: A Methodological Proposal","volume":"17","author":"Ferreira","year":"2011","journal-title":"Univers. Comput. Sci."},{"key":"ref_51","doi-asserted-by":"crossref","first-page":"64","DOI":"10.1016\/j.jclepro.2012.07.023","article-title":"Social learning indicators in sustainability-oriented regional learning networks","volume":"49","author":"Barton","year":"2013","journal-title":"J. Clean. Prod."},{"key":"ref_52","doi-asserted-by":"crossref","unstructured":"Hazelkorn, E., Coates, H., and McCormick, A.C. (2018). Research Handbook on Quality, Performance and Accountability in Higher Education, Edward Elgar Publishing.","DOI":"10.4337\/9781785369759"},{"key":"ref_53","doi-asserted-by":"crossref","first-page":"672","DOI":"10.1108\/17410401211249210","article-title":"Framework for performance measurement and management in a collaborative business environment","volume":"61","author":"Ferreira","year":"2012","journal-title":"Int. J. Product. Perform. Manag."},{"key":"ref_54","doi-asserted-by":"crossref","first-page":"305","DOI":"10.1111\/j.1468-2370.2011.00318.x","article-title":"Performance measurement: Challenges for tomorrow","volume":"14","author":"Bititci","year":"2012","journal-title":"Int. J. Manag. Rev."},{"key":"ref_55","first-page":"415","article-title":"A framework for network level performance measurement system in SME networks","volume":"3","author":"Vuorinen","year":"2008","journal-title":"Int. J. Netw. Virtual Organ."},{"key":"ref_56","doi-asserted-by":"crossref","first-page":"212","DOI":"10.1504\/MEJM.2015.072466","article-title":"Collaborative networks\u2019 performance index","volume":"2","author":"Adel","year":"2015","journal-title":"Middle East J. Manag."},{"key":"ref_57","doi-asserted-by":"crossref","first-page":"592","DOI":"10.1080\/09537280701546880","article-title":"Performance indicators for collaborative networks based on collaboration benefits","volume":"18","author":"Abreu","year":"2007","journal-title":"Prod. Plan. Control."},{"key":"ref_58","doi-asserted-by":"crossref","first-page":"2021","DOI":"10.5267\/j.msl.2014.8.019","article-title":"Key indicators for organizational performance measurement","volume":"4","author":"Haddadi","year":"2014","journal-title":"Manag. Sci. Lett."},{"key":"ref_59","doi-asserted-by":"crossref","unstructured":"Peffers, K., Tuunanen, T., Gengler, C.E., Rossi, M., Hui, W., Virtanen, V., and Bragge, J. (2006, January 24\u201325). The Design Science Research Process: A Model for Producing and Presenting Information Systems Research. Proceedings of the 1st International Conference on Design Science Research in Information Systems and Technology, Claremont, CA, USA.","DOI":"10.2753\/MIS0742-1222240302"},{"key":"ref_60","doi-asserted-by":"crossref","first-page":"142","DOI":"10.1016\/j.accinf.2011.02.004","article-title":"A design science research methodology and its application to accounting information systems research","volume":"12","author":"Geerts","year":"2011","journal-title":"Int. J. Account. Inf. Syst."},{"key":"ref_61","doi-asserted-by":"crossref","unstructured":"(2022, May 10). ed-en hub. Available online: http:\/\/edenhub.eu\/.","DOI":"10.3221\/IGF-ESIS.37.10"},{"key":"ref_62","doi-asserted-by":"crossref","first-page":"982","DOI":"10.1287\/mnsc.35.8.982","article-title":"User Acceptance of Computer Technology: A Comparison of Two Theoretical Models","volume":"35","author":"Davis","year":"1989","journal-title":"Manag. Sci."},{"key":"ref_63","doi-asserted-by":"crossref","unstructured":"Cress, U., Moskaliuk, J., and Jeong, H. (2016). Mass Collaboration and Education, Springer.","DOI":"10.1007\/978-3-319-13536-6"},{"key":"ref_64","doi-asserted-by":"crossref","unstructured":"Richardson, M., and Domingos, P. (2003, January 23\u201325). Building Large Knowledge Bases by Mass Collaboration. Proceedings of the International Conference on Knowledge Capture\u2014K-CAP \u201903, Sanibel Island, FL, USA.","DOI":"10.1145\/945645.945665"},{"key":"ref_65","doi-asserted-by":"crossref","unstructured":"Zamiri, M., Camarinha-Matos, L.M., and Sarraipa, J. (2022). Meta-Governance Framework to Guide the Establishment of Mass Collaborative Learning Communities. Computers, 11.","DOI":"10.3390\/computers11010012"}],"container-title":["Applied Sciences"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2076-3417\/12\/16\/8356\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T00:13:09Z","timestamp":1760141589000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2076-3417\/12\/16\/8356"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,8,21]]},"references-count":65,"journal-issue":{"issue":"16","published-online":{"date-parts":[[2022,8]]}},"alternative-id":["app12168356"],"URL":"https:\/\/doi.org\/10.3390\/app12168356","relation":{},"ISSN":["2076-3417"],"issn-type":[{"type":"electronic","value":"2076-3417"}],"subject":[],"published":{"date-parts":[[2022,8,21]]}}}