{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,29]],"date-time":"2026-05-29T15:06:54Z","timestamp":1780067214392,"version":"3.54.0"},"reference-count":72,"publisher":"MDPI AG","issue":"1","license":[{"start":{"date-parts":[[2021,1,26]],"date-time":"2021-01-26T00:00:00Z","timestamp":1611619200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/100000001","name":"National Science Foundation","doi-asserted-by":"publisher","award":["1721003"],"award-info":[{"award-number":["1721003"]}],"id":[{"id":"10.13039\/100000001","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Systems"],"abstract":"<jats:p>Many researchers have identified the need for a more holistic understanding of the role of feedback in supporting learning in online environments. This study explores how our design, development, and implementation of an online feedback facilitation system influenced high school science teachers\u2019 learning in an asynchronous teacher professional development online course. We then describe teachers\u2019 and facilitators\u2019, i.e., feedback providers\u2019, perceptions of the effectiveness of the system\u2019s features for supporting participants\u2019 learning and engagement. Our work also responds to recent calls for developing a more nuanced understanding of how the complexity of feedback influences learning and the need for more qualitative research on online facilitators\u2019 and learners\u2019 experiences working with new technologies. Results demonstrated that, despite the difficulty of analyzing the complex variables influencing learners\u2019 interactions and perceptions of the feedback system, designing adaptive feedback systems that draw on the principles of design-based implementation research (DBIR) offer promise for enhancing the systems\u2019 contributions to teacher learning.<\/jats:p>","DOI":"10.3390\/systems9010010","type":"journal-article","created":{"date-parts":[[2021,1,26]],"date-time":"2021-01-26T12:03:57Z","timestamp":1611662637000},"page":"10","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":12,"title":["Designing Feedback Systems: Examining a Feedback Approach to Facilitation in an Online Asynchronous Professional Development Course for High School Science Teachers"],"prefix":"10.3390","volume":"9","author":[{"given":"Amin","family":"Marei","sequence":"first","affiliation":[{"name":"Graduate School of Education, University of Pennsylvania, Philadelphia, PA 19104, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Susan A.","family":"Yoon","sequence":"additional","affiliation":[{"name":"Graduate School of Education, University of Pennsylvania, Philadelphia, PA 19104, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jae-Un","family":"Yoo","sequence":"additional","affiliation":[{"name":"Graduate School of Education, University of Pennsylvania, Philadelphia, PA 19104, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Thomas","family":"Richman","sequence":"additional","affiliation":[{"name":"Graduate School of Education, University of Pennsylvania, Philadelphia, PA 19104, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Noora","family":"Noushad","sequence":"additional","affiliation":[{"name":"Graduate School of Education, University of Pennsylvania, Philadelphia, PA 19104, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Katherine","family":"Miller","sequence":"additional","affiliation":[{"name":"Graduate School of Education, University of Pennsylvania, Philadelphia, PA 19104, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jooeun","family":"Shim","sequence":"additional","affiliation":[{"name":"Graduate School of Education, University of Pennsylvania, Philadelphia, PA 19104, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2021,1,26]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"247","DOI":"10.1080\/00461520.2016.1158654","article-title":"Seven affordances of computer-supported collaborative learning: How to support collaborative learning? How can technologies help?","volume":"51","author":"Jeong","year":"2016","journal-title":"Educ. Psychol."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"799","DOI":"10.3102\/0034654318791584","article-title":"The role of collaboration, computer use, learning environments, and supporting strategies in CSCL: A meta-analysis","volume":"88","author":"Chen","year":"2018","journal-title":"Rev. Educ. Res."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Scardamalia, M., and Bereiter, C. (2014). Smart technology for self-organizing processes. Smart Learn. Environ., 1.","DOI":"10.1186\/s40561-014-0001-8"},{"key":"ref_4","first-page":"415","article-title":"Reflections and looking ahead for CSCL: Digital infrastructures, digital tools, and collaborative learning","volume":"14","author":"Ludvigsen","year":"2019","journal-title":"Int. J. Comput. Collab. Learn."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"251","DOI":"10.1007\/s11412-019-09310-5","article-title":"Real-time orchestrational technologies in computer-supported collaborative learning: An introduction to the special issue","volume":"14","author":"Matuk","year":"2019","journal-title":"Int. J. Comput. Supported Collab. Learn."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"932","DOI":"10.1016\/j.ejor.2017.11.006","article-title":"Applying systems thinking models of organizational design and change in community operational research","volume":"268","author":"Cabrera","year":"2018","journal-title":"Eur. J. Oper. Res."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"68","DOI":"10.1080\/02602938.2020.1733490","article-title":"How conceptualising respect can inform feedback pedagogies","volume":"46","author":"Zhou","year":"2020","journal-title":"Assess. Eval. High. Educ."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"178","DOI":"10.1080\/01587919.2014.917707","article-title":"Case study: Using MOOCs for conventional college coursework","volume":"35","author":"Firmin","year":"2014","journal-title":"Distance Educ."},{"key":"ref_9","first-page":"1","article-title":"A study on the development of a MOOC design model","volume":"17","author":"Gayoung","year":"2016","journal-title":"Educ. Technol. Int."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"9","DOI":"10.1016\/j.compedu.2018.02.013","article-title":"Learning engagement and persistence in massive open online courses (MOOCS)","volume":"122","author":"Jung","year":"2018","journal-title":"Comput. Educ."},{"key":"ref_11","unstructured":"Siemens, G., Downes, S., Cormier, D., and Kop, R. (2020, October 17). PLENK 2010\u2013Personal Learning Environments, Networks and Knowledge. Available online: https:\/\/www.islandscholar.ca\/islandora\/object\/ir%3A20500."},{"key":"ref_12","first-page":"845","article-title":"A Framework towards educational scalability of open online courses","volume":"23","author":"Kasch","year":"2017","journal-title":"J. Univers. Comput. Sci."},{"key":"ref_13","first-page":"104042","article-title":"Stepping back and stepping in: Facilitating learner-centered experiences in MOOCs","volume":"160","author":"Yurkofsky","year":"2020","journal-title":"Comput. Educ."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"790","DOI":"10.1080\/09650792.2016.1245626","article-title":"Comparing face-to-face and asynchronous online communication as mechanisms for critical reflective dialogue","volume":"25","author":"Salter","year":"2017","journal-title":"Educ. Action Res."},{"key":"ref_15","first-page":"33","article-title":"US participants\u2019 perceptions of online professional development","volume":"82","author":"Parsons","year":"2019","journal-title":"Teach. Particip. Educ. Int. J. Res. Stud."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Webb, D.C., Nickerson, H., and Bush, J.B. (2017, January 8\u201311). A comparative analysis of online and face-to-face professional development models for CS education. Proceedings of the SIGCSE \u201817: The 48th ACM Technical Symposium on Computer Science Education, Seattle, WA, USA.","DOI":"10.1145\/3017680.3017784"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"351","DOI":"10.1007\/s11412-020-09326-2","article-title":"Encouraging collaboration and building community in online asynchronous professional development: Designing for social capital","volume":"15","author":"Yoon","year":"2020","journal-title":"Int. J. Comput. Supported Collab. Learn."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"103736","DOI":"10.1016\/j.compedu.2019.103736","article-title":"Design principles for fostering pedagogical provenance through research in technology supported learning","volume":"146","author":"Turvey","year":"2020","journal-title":"Comput. Educ."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"505","DOI":"10.2105\/AJPH.2005.066043","article-title":"Learning from evidence in a complex world","volume":"96","author":"Sterman","year":"2006","journal-title":"Am. J. Public Health"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"130","DOI":"10.1016\/j.compedu.2019.04.003","article-title":"Elaborated feedback and learning: Examining cognitive and motivational influences","volume":"136","author":"Wang","year":"2019","journal-title":"Comput. Educ."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"1524","DOI":"10.1002\/tea.21336","article-title":"Early elementary students\u2019 understanding of complex ecosystems: A learning progression approach","volume":"53","author":"Hokayem","year":"2016","journal-title":"J. Res. Sci. Teach."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Scott, W.R., and Davis, G.F. (2015). Organizations and Organizing: Rational, Natural and Open Systems Perspectives, Routledge. [1st ed.].","DOI":"10.4324\/9781315663371"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"263","DOI":"10.1080\/02602930701292548","article-title":"Feedback on assessment: Students\u2019 perceptions of quality and effectiveness","volume":"33","author":"Lizzio","year":"2008","journal-title":"Assess. Eval. High. Educ."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"1237","DOI":"10.1080\/02602938.2019.1599815","article-title":"The challenges of feedback in higher education","volume":"44","author":"Henderson","year":"2019","journal-title":"Assess. Eval. High. Educ."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"54","DOI":"10.1108\/IJSHE-09-2014-0127","article-title":"Practice makes pedagogy\u2013John Dewey and skills-based sustainability education","volume":"17","author":"Tarrant","year":"2016","journal-title":"Int. J. Sustain. High. Educ."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"428","DOI":"10.1080\/02602938.2017.1356905","article-title":"A feedback model for data-rich learning experiences","volume":"43","author":"Pardo","year":"2018","journal-title":"Assess. Eval. High. Educ."},{"key":"ref_27","unstructured":"Sch\u00f6n, D., and Argyris, C. (1996). Organizational learning II: Theory, Method and Practice, Addison-Wesley Publishing Company."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"60","DOI":"10.1016\/j.compedu.2017.03.015","article-title":"Analyzing gameplay data to inform feedback loops in The Radix Endeavor","volume":"111","author":"Cheng","year":"2017","journal-title":"Comput. Educ."},{"key":"ref_29","first-page":"30","article-title":"Feedback: Part of a system","volume":"70","author":"Wiliam","year":"2012","journal-title":"Educ. Leadersh."},{"key":"ref_30","first-page":"40","article-title":"System dynamics perspectives and modeling opportunities for research in operations management","volume":"39","author":"Sterman","year":"2015","journal-title":"J. Oper. Manag."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"705","DOI":"10.1080\/02602938.2018.1531108","article-title":"Feedback loops and the longer-term: Towards feedback spirals","volume":"44","author":"Carless","year":"2019","journal-title":"Assess. Eval. High. Educ."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"291","DOI":"10.1002\/sdr.4260100214","article-title":"Learning in and about complex systems","volume":"10","author":"Sterman","year":"1994","journal-title":"Syst. Dyn. Rev."},{"key":"ref_33","first-page":"30","article-title":"Two loops that need closing: Contingent faculty perceptions of outcomes assessment","volume":"64","author":"Scott","year":"2015","journal-title":"J. Gen. Educ."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"744","DOI":"10.1007\/s11606-019-04874-2","article-title":"Feedback redefined: Principles and practice","volume":"34","author":"Ramani","year":"2019","journal-title":"J. Gen. Intern. Med."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"52","DOI":"10.1016\/j.iheduc.2018.01.003","article-title":"Student perception of helpfulness of facilitation strategies that enhance instructor presence, connectedness, engagement and learning in online courses","volume":"37","author":"Martin","year":"2018","journal-title":"Internet High. Educ."},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Davis, D., Jivet, I., Kizilcec, R.F., Chen, G., Hauff, C., and Houben, G.J. (2017, January 13\u201317). Follow the successful crowd: Raising MOOC completion rates through social comparison at scale. Proceedings of the LAK\u201917 conference proceeding: The Seventh International Learning Analytics & Knowledge Conference, Simon Fraser University, Vancouver, BC, Canada.","DOI":"10.1145\/3027385.3027411"},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"1505","DOI":"10.1007\/s11423-018-9620-z","article-title":"The effects of an online professional development course on participants\u2019 perceptions, attitudes, self-efficacy, and behavioral intentions regarding digital game-based learning","volume":"66","author":"An","year":"2018","journal-title":"Educ. Technol. Res. Dev."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"134","DOI":"10.24059\/olj.v20i2.775","article-title":"Exploring the relationships between facilitation methods, students\u2019 sense of community, and their online behaviors","volume":"20","author":"Phirangee","year":"2016","journal-title":"Online Learn."},{"key":"ref_39","first-page":"203","article-title":"Instructor experiences designing MOOCs in higher education: Pedagogical, resource, and logistical considerations and challenges","volume":"22","author":"Zhu","year":"2018","journal-title":"Online Learn."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"19","DOI":"10.1007\/s11251-014-9329-2","article-title":"Student perceptions of peer versus instructor facilitation of asynchronous online discussions: Further findings from three cases","volume":"43","author":"Hew","year":"2015","journal-title":"Instr. Sci."},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"24","DOI":"10.1016\/j.edurev.2016.10.003","article-title":"Exploring the value of peer feedback in online learning for the provider","volume":"20","author":"Kral","year":"2017","journal-title":"Educ. Res. Rev."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"285","DOI":"10.1080\/13562517.2012.719154","article-title":"The feedback triangle and the enhancement of dialogic feedback processes","volume":"18","author":"Yang","year":"2013","journal-title":"Teach. High. Educ."},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"899","DOI":"10.1080\/02602938.2013.875985","article-title":"Closing the feedback loop? Iterative feedback between tutor and student in coursework assessments","volume":"39","author":"Barker","year":"2014","journal-title":"Assess. Eval. High. Educ."},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"408","DOI":"10.3102\/0034654311415121","article-title":"Professional development for technology-enhanced inquiry science","volume":"81","author":"Gerard","year":"2011","journal-title":"Rev. Educ. Res."},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"99","DOI":"10.1007\/s11251-016-9388-7","article-title":"Teaching about complex systems is no simple matter: Building effective professional development for computer-supported complex systems instruction","volume":"45","author":"Yoon","year":"2017","journal-title":"Instr. Sci."},{"key":"ref_46","first-page":"136","article-title":"Design-based implementation research: An emerging model for transforming the relationship of research and practice","volume":"112","author":"Fishman","year":"2013","journal-title":"Natl. Soc. Study Educ."},{"key":"ref_47","doi-asserted-by":"crossref","unstructured":"Fishman, B.J., and Penuel, W.R. (2018). Design-Based Implementation Research. International Handbook of the Learning Sciences, Routledge.","DOI":"10.4324\/9781315617572-38"},{"key":"ref_48","doi-asserted-by":"crossref","unstructured":"Hew, K.F., Qiao, C., and Tang, Y. (2018). Understanding student engagement in large-scale open online courses: A machine learning facilitated analysis of student\u2019s reflections in 18 highly rated MOOCs. Int. Rev. Res. Open Distrib. Learn., 19.","DOI":"10.19173\/irrodl.v19i3.3596"},{"key":"ref_49","doi-asserted-by":"crossref","first-page":"501","DOI":"10.1080\/02602931003786559","article-title":"From monologue to dialogue: Improving written feedback processes in mass higher education","volume":"35","author":"Nicol","year":"2010","journal-title":"Assess. Eval. High. Educ."},{"key":"ref_50","doi-asserted-by":"crossref","first-page":"309","DOI":"10.1007\/s11412-015-9219-x","article-title":"Group-level formative feedback and metadiscourse","volume":"10","author":"Resendes","year":"2015","journal-title":"Int. J. Comput. Supported Collab. Learn."},{"key":"ref_51","doi-asserted-by":"crossref","unstructured":"Kizilcec, R.F., and Halawa, S. (2015, January 14\u201318). Attrition and achievement gaps in online learning. Proceedings of the Second (2015) ACM Conference on Learning@ Scale, Vancouver, BC, Canada.","DOI":"10.1145\/2724660.2724680"},{"key":"ref_52","doi-asserted-by":"crossref","unstructured":"Kizilcec, R.F., P\u00e9rez-Sanagust\u00edn, M., and Maldonado, J.J. (2016, January 25\u201326). Recommending self-regulated learning strategies does not improve performance in a MOOC. Proceedings of the third (2016) ACM conference on learning@ scale, Edinburgh, Scotland, UK.","DOI":"10.1145\/2876034.2893378"},{"key":"ref_53","doi-asserted-by":"crossref","first-page":"591","DOI":"10.1016\/j.chb.2016.07.038","article-title":"Participant self-efficacy, academic self-efficacy, and computer self-efficacy as predictors of attitude toward applying computer-supported education","volume":"64","author":"Akan","year":"2016","journal-title":"Comput. Hum. Behav."},{"key":"ref_54","doi-asserted-by":"crossref","unstructured":"Darling-Hammond, L., Hyler, M.E., and Gardner, M. (2017). Effective Participant Professional Development, Learning Policy Institute.","DOI":"10.54300\/122.311"},{"key":"ref_55","doi-asserted-by":"crossref","unstructured":"Baroody, A.J., and Dowker, A. (2013). ; The Development of Arithmetic Concepts and Skills: Constructive Adaptive Expertise, Routledge.","DOI":"10.4324\/9781410607218"},{"key":"ref_56","first-page":"108","article-title":"Prospective participants development of adaptive expertise","volume":"49","author":"Anthony","year":"2015","journal-title":"Teach. Particip. Educ."},{"key":"ref_57","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1007\/s11423-019-09701-3","article-title":"Cognitive load theory and educational technology","volume":"68","author":"Sweller","year":"2020","journal-title":"Educ. Technol. Res. Dev."},{"key":"ref_58","first-page":"1064","article-title":"Connectivism: A knowledge learning theory for the digital age?","volume":"38","author":"Goldie","year":"2016","journal-title":"Med. Particip."},{"key":"ref_59","first-page":"1","article-title":"The moderating effect of social capital on co-regulated learning for MOOC achievement","volume":"5","author":"Chaker","year":"2020","journal-title":"Educ. Inf. Technol."},{"key":"ref_60","doi-asserted-by":"crossref","first-page":"271","DOI":"10.1007\/s10490-012-9330-8","article-title":"Managerial ties, organizational learning, and opportunity capture: A social capital perspective","volume":"31","author":"Li","year":"2014","journal-title":"Asia Pac. J. Manag."},{"key":"ref_61","unstructured":"Adamopoulos, P. (2013, January 15\u201318). What makes a great MOOC? An interdisciplinary analysis of student retention in online courses. Proceedings of the International Conference on Information Systems, ICIS 2013, Milano, Italy."},{"key":"ref_62","doi-asserted-by":"crossref","first-page":"528","DOI":"10.1111\/bjet.12275","article-title":"Survey of learning experiences and influence of learning style preferences on user intentions regarding MOOC s","volume":"46","author":"Chang","year":"2015","journal-title":"Br. J. Educ. Technol."},{"key":"ref_63","doi-asserted-by":"crossref","first-page":"660","DOI":"10.1007\/s40593-016-0107-y","article-title":"Technology support for discussion based learning: From computer supported collaborative learning to the future of massive open online courses","volume":"26","author":"Ferschke","year":"2016","journal-title":"Int. J. Artif. Intell. Educ."},{"key":"ref_64","doi-asserted-by":"crossref","unstructured":"Glaser, B.G., and Strauss, A.L. (2017). Discovery of Grounded Theory: Strategies for Qualitative Research, Routledge.","DOI":"10.4324\/9780203793206"},{"key":"ref_65","doi-asserted-by":"crossref","first-page":"294","DOI":"10.1177\/0049124113500475","article-title":"Coding in-depth semistructured interviews: Problems of unitization and intercoder reliability and agreement","volume":"42","author":"Campbell","year":"2013","journal-title":"Sociol. Methods Res."},{"key":"ref_66","doi-asserted-by":"crossref","unstructured":"Putnam, R. (2000). Bowling Alone, Simon & Schuster.","DOI":"10.1145\/358916.361990"},{"key":"ref_67","doi-asserted-by":"crossref","first-page":"281","DOI":"10.1080\/01587919.2018.1476844","article-title":"Investigating students\u2019 perceptions of instructional strategies to establish social presence","volume":"39","author":"Lowenthal","year":"2018","journal-title":"Distance Educ."},{"key":"ref_68","first-page":"83","article-title":"Reading activities in blended learning: Recommendations for university language preparatory course participants","volume":"5","author":"Ulker","year":"2019","journal-title":"Int. J. Soc. Sci. Educ. Stud."},{"key":"ref_69","first-page":"197","article-title":"Educators\u2019 preparation to teach, perceived teaching presence, and perceived teaching presence behaviors in blended and online learning environments","volume":"22","author":"Gurley","year":"2018","journal-title":"Online Learn."},{"key":"ref_70","unstructured":"Hollands, F.M., and Tirthali, D. (2014). MOOCs: Expectations and reality. Center for Benefit-Cost Studies of Education, Teachers College, Columbia University."},{"key":"ref_71","doi-asserted-by":"crossref","first-page":"1273","DOI":"10.1007\/s11423-016-9457-2","article-title":"Instructional design, facilitation, and perceived learning outcomes: An exploratory case study of a human trafficking MOOC for attitudinal change","volume":"64","author":"Watson","year":"2016","journal-title":"Educ. Technol. Res. Dev."},{"key":"ref_72","doi-asserted-by":"crossref","first-page":"682","DOI":"10.1007\/s10956-014-9494-8","article-title":"An investigation of university students\u2019 collaborative inquiry learning behaviors in an augmented reality simulation and a traditional simulation","volume":"23","author":"Wang","year":"2014","journal-title":"J. Sci. Educ. Technol."}],"container-title":["Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2079-8954\/9\/1\/10\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T05:15:43Z","timestamp":1760159743000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2079-8954\/9\/1\/10"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,1,26]]},"references-count":72,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2021,3]]}},"alternative-id":["systems9010010"],"URL":"https:\/\/doi.org\/10.3390\/systems9010010","relation":{},"ISSN":["2079-8954"],"issn-type":[{"value":"2079-8954","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,1,26]]}}}