{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,3]],"date-time":"2026-06-03T17:47:11Z","timestamp":1780508831812,"version":"3.54.1"},"reference-count":61,"publisher":"MDPI AG","issue":"2","license":[{"start":{"date-parts":[[2021,4,5]],"date-time":"2021-04-05T00:00:00Z","timestamp":1617580800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/100006807","name":"Furman University","doi-asserted-by":"publisher","award":["Furman Advantage"],"award-info":[{"award-number":["Furman Advantage"]}],"id":[{"id":"10.13039\/100006807","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Systems"],"abstract":"<jats:p>The system that shapes a problem can be represented using a map, in which relevant constructs are listed as nodes, and salient interrelationships are provided as directed edges which track the direction of causation. Such representations are particularly useful to address complex problems which are multi-factorial and may involve structures such as loops, in contrast with simple problems which may have a clear root cause and a short chain of causes-and-effects. Although students are often evaluated based on either simple problems or simplified situations (e.g., true\/false, multiple choice), they need systems thinking skills to eventually deal with complex, open-ended problems in their professional lives. A starting point is thus to construct a representation of the problem space, such as a causal map, and then to identify and contrast solutions by navigating this map. The initial step of abstracting a system into a map is challenging for students: unlike seasoned experts, they lack a detailed understanding of the application domain, and hence struggle in capturing its key concepts and interrelationships. Case libraries can remedy this disadvantage, as they can transfer the knowledge of experts to novices. However, the content of the cases can impact the perspectives of students. For example, their understanding of a system (as reflected in a map) may differ when they are exposed to case studies depicting successful or failed interventions in a system. Previous studies have abundantly documented that cases can support students, using a variety of metrics such as test scores. In the present study, we examine the ways in which the representation of a system (captured as a causal map) changes as a function of exposure to certain types of evidence. Our experiments across three cohorts at two institutions show that providing students with cases tends to broaden their coverage of the problem space, but the knowledge afforded by the cases is integrated in the students\u2019 maps differently depending on the type of case, as well as the cohort of students.<\/jats:p>","DOI":"10.3390\/systems9020023","type":"journal-article","created":{"date-parts":[[2021,4,5]],"date-time":"2021-04-05T11:48:29Z","timestamp":1617623309000},"page":"23","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":11,"title":["How Perspectives of a System Change Based on Exposure to Positive or Negative Evidence"],"prefix":"10.3390","volume":"9","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-6816-355X","authenticated-orcid":false,"given":"Philippe J.","family":"Giabbanelli","sequence":"first","affiliation":[{"name":"Department of Computer Science and Software Engineering, Miami University, Oxford, OH 45056, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Andrew A.","family":"Tawfik","sequence":"additional","affiliation":[{"name":"Department of Instructional Design and Technology, University of Memphis, Memphis, TN 38152, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2021,4,5]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"232","DOI":"10.1016\/j.envsoft.2018.08.028","article-title":"Tools and methods in participatory modeling: Selecting the right tool for the job","volume":"109","author":"Voinov","year":"2018","journal-title":"Environ. Model. Softw."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Walsh, R. (2018). Narrative theory for complexity scientists. Narrating Complexity, Springer.","DOI":"10.1007\/978-3-319-64714-2"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"1285","DOI":"10.1177\/0042098013505653","article-title":"Narrating resilience: Transforming urban systems through collaborative storytelling","volume":"52","author":"Wessells","year":"2015","journal-title":"Urban. Stud."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"31","DOI":"10.1002\/sres.2121","article-title":"Contemporary iconography for rich picture construction","volume":"30","author":"Berg","year":"2013","journal-title":"Syst. Res. Behav. Sci."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"El-Sayed, A.M., and Galea, S. (2017). Mapping Complex Systems of Population Health. Systems Science and Population Health, Oxford University Press.","DOI":"10.1093\/acprof:oso\/9780190492397.001.0001"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Bala, B.K., Arshad, F.M., and Noh, K.M. (2017). Causal loop diagrams. System Dynamics, Springer.","DOI":"10.1007\/978-981-10-2045-2"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Moon, B., Hoffman, R.R., Novak, J., and Canas, A. (2011). Applied Concept Mapping: Capturing, Analyzing, and Organizing Knowledge, CRC Press.","DOI":"10.1201\/b10716"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Gray, S., Sterling, E.J., Aminpour, P., Goralnik, L., Singer, A., Wei, C., Akabas, S., Jordan, R.C., Giabbanelli, P.J., and Hodbod, J. (2019). Assessing (social-ecological) systems thinking by evaluating cognitive maps. Sustainability, 11.","DOI":"10.3390\/su11205753"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Reddy, T, Giabbanelli, P.J, and Mago, V.K (2019, January 26). The artificial facilitator: Guiding participants in developing causal maps using voice-activated technologies. Proceedings of the International Conference on Human-Computer Interaction, Copenhagen, Denmark.","DOI":"10.1007\/978-3-030-22419-6_9"},{"key":"ref_10","unstructured":"Lavin, E.A., Giabbanelli, P.J., Stefanik, A.T., Gray, S.A., and Arlinghaus, R. (2018, January 15). Should we simulate mental models to assess whether they agree?. Proceedings of the Annual Simulation Symposium, Baltimore, MD, USA."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.jort.2015.09.001","article-title":"The structure and function of angler mental models about fish population ecology: The influence of specialization and target species","volume":"12","author":"Gray","year":"2015","journal-title":"J. Outdoor Recreat. Tour."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.ejrh.2016.01.035","article-title":"Using mental-modelling to explore how irrigators in the Murray\u2013Darling Basin make water-use decisions","volume":"6","author":"Douglas","year":"2016","journal-title":"J. Hydrol. Reg. Stud."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"S12","DOI":"10.1016\/j.jcjd.2015.01.058","article-title":"Exploring the interactions between physical well-being, and obesity","volume":"39","author":"Drasic","year":"2015","journal-title":"Can. J. Diabetes"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"S13","DOI":"10.1038\/oby.2009.426","article-title":"Implications of the foresight obesity system map for solutions to childhood obesity","volume":"18","author":"Finegood","year":"2010","journal-title":"Obesity"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Rahimi, N., Jetter, A.J., Weber, C.M., and Wild, K. (2018). Soft Data analytics with fuzzy cognitive maps: Modeling health technology adoption by elderly women. Advanced Data Analytics in Health, Springer.","DOI":"10.1007\/978-3-319-77911-9_4"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"443","DOI":"10.1007\/s11423-013-9297-2","article-title":"Validation study of a method for assessing complex ill-structured problem solving by using causal representations","volume":"61","author":"Eseryel","year":"2013","journal-title":"Educ. Technol. Res. Dev."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"41","DOI":"10.1007\/s11251-009-9097-6","article-title":"The mystery of cognitive structure and how we can detect it: Tracking the development of cognitive structures over time","volume":"39","author":"Ifenthaler","year":"2011","journal-title":"Instr. Sci."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Trumpower, D.L., Filiz, M., and Sarwar, G.S. (2014). Assessment for learning using digital knowledge maps. Digital Knowledge Maps in Education, Springer.","DOI":"10.1007\/978-1-4614-3178-7_12"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Giabbanelli, P.J., and Tawfik, A.A. (2020, January 19\u201324). Reducing the Gap Between the Conceptual Models of Students and Experts Using Graph-Based Adaptive Instructional Systems. Proceedings of the International Conference on Human-Computer Interaction, Copenhagen, Denmark.","DOI":"10.1007\/978-3-030-60128-7_40"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"161","DOI":"10.1007\/s10758-017-9338-8","article-title":"Overcoming the PBL assessment challenge: Design and development of the incremental thesaurus for assessing causal maps (ITACM)","volume":"24","author":"Giabbanelli","year":"2019","journal-title":"Technol. Knowl. Learn."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Giabbanelli, P.J., Tawfik, A.A., and Gupta, V.K. (2019). Learning analytics to support teachers\u2019 assessment of problem solving: A novel application for machine learning and graph algorithms. Util. Learn. Anal. Support Study Success, 175\u2013199.","DOI":"10.1007\/978-3-319-64792-0_11"},{"key":"ref_22","first-page":"5","article-title":"Creating a learning space in problem-based learning","volume":"7","year":"2013","journal-title":"Interdiscip. J. Probl. Based Learn."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Jonassen, D.H. (2010). Learning to Solve Problems: A Handbook for Designing Problem-Solving Learning Environments. Routledge.","DOI":"10.4324\/9780203847527"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"466","DOI":"10.1080\/19320240903346448","article-title":"Places to intervene to make complex food systems more healthy, green, fair, and affordable","volume":"4","author":"Malhi","year":"2009","journal-title":"J. Hunger Environ. Nutr."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"307","DOI":"10.1207\/s15327809jls0203_3","article-title":"Constraining novices to perform expert-like problem analyses: Effects on schema acquisition","volume":"2","author":"Dufresne","year":"1992","journal-title":"J. Learn. Sci."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Jeong, A. (2014). Sequentially analyzing and modeling causal mapping processes that support causal understanding and systems thinking. Digital Knowledge Maps in Education, Springer.","DOI":"10.1007\/978-1-4614-3178-7_13"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"104113","DOI":"10.1016\/j.compedu.2020.104113","article-title":"Modeling the relationship between students\u2019 prior knowledge, causal reasoning processes, and quality of causal maps","volume":"163","author":"Shin","year":"2021","journal-title":"Comput. Educ."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"307","DOI":"10.1080\/10508400701413401","article-title":"Fish swim, rocks sit, and lungs breathe: Expert-novice understanding of complex systems","volume":"16","author":"Marathe","year":"2007","journal-title":"J. Learn. Sci."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"202","DOI":"10.1007\/s11528-019-00453-8","article-title":"The utility of design thinking to promote systemic instructional design practices in the workplace","volume":"64","author":"Stefaniak","year":"2020","journal-title":"TechTrends"},{"key":"ref_30","first-page":"1","article-title":"Analysis of problem decomposition strategies of novice industrial designers using network-based cognitive maps","volume":"25","author":"Zhou","year":"2021","journal-title":"Int. J. Technol. Des. Educ."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"459","DOI":"10.1016\/j.chb.2018.03.018","article-title":"Supports for deeper learning of inquiry-based ecosystem science in virtual environments-Comparing virtual and physical concept mapping","volume":"87","author":"Metcalf","year":"2018","journal-title":"Comput. Hum. Behav."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.compedu.2018.11.002","article-title":"Digital games as sources for science analogies: Learning about energy through play","volume":"130","author":"Martin","year":"2019","journal-title":"Comput. Educ."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"103731","DOI":"10.1016\/j.compedu.2019.103731","article-title":"Exploring student information problem solving behaviour using fine-grained concept map and search tool data","volume":"145","author":"Laan","year":"2020","journal-title":"Comput. Educ."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"e0235209","DOI":"10.1371\/journal.pone.0235209","article-title":"Assisting students\u2019 writing with computer-based concept map feedback: A validation study of the CohViz feedback system","volume":"15","author":"Burkhart","year":"2020","journal-title":"PLoS ONE"},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"459","DOI":"10.1016\/j.chb.2017.03.008","article-title":"Formative computer-based feedback in the university classroom: Specific concept maps scaffold students\u2019 writing","volume":"72","author":"Lachner","year":"2017","journal-title":"Comput. Hum. Behav."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"23","DOI":"10.1007\/s10758-017-9324-1","article-title":"How novices use expert case libraries for problem solving","volume":"24","author":"Tawfik","year":"2019","journal-title":"Technol. Knowl. Learn."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"103","DOI":"10.1046\/j.0266-4909.2002.00010.x","article-title":"The effects of case libraries on problem solving","volume":"19","author":"Jonassen","year":"2003","journal-title":"J. Comput. Assist. Learn."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"177","DOI":"10.1007\/s10758-017-9335-y","article-title":"Using a Recommendation System to Support Problem Solving and Case-Based Reasoning Retrieval","volume":"23","author":"Tawfik","year":"2018","journal-title":"Technol. Knowl. Learn."},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Schank, R.C. (1999). Dynamic Memory Revisited. Cambridge University Press.","DOI":"10.1017\/CBO9780511527920"},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Giabbanelli, P.J., and Baniukiewicz, M. (2018). Navigating complex systems for policymaking using simple software tools. Advanced Data Analytics in Health, Springer.","DOI":"10.1007\/978-3-319-77911-9_2"},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Shute, V.J., and Zapata-Rivera, D. (2008). Using an evidence-based approach to assess mental models. Understanding Models for Learning and Instruction, Springer.","DOI":"10.1007\/978-0-387-76898-4_2"},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"46136","DOI":"10.1109\/ACCESS.2019.2908622","article-title":"Identifying the components and interrelationships of smart cities in Indonesia: Supporting policymaking via fuzzy cognitive systems","volume":"7","author":"Firmansyah","year":"2019","journal-title":"IEEE Access"},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Brandes, U. (2005). Network Analysis: Methodological Foundations. Springer Science & Business Media.","DOI":"10.1007\/b106453"},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"135","DOI":"10.1016\/j.evalprogplan.2018.03.003","article-title":"Mind maps and network analysis to evaluate conceptualization of complex issues: A case example evaluating systems science workshops for childhood obesity prevention","volume":"68","author":"Frerichs","year":"2018","journal-title":"Eval. Program Plan."},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"33","DOI":"10.1080\/00219266.2018.1541001","article-title":"Using concept maps to characterise cellular respiration knowledge in undergraduate students","volume":"54","author":"Galt","year":"2020","journal-title":"J. Biol. Educ."},{"key":"ref_46","doi-asserted-by":"crossref","unstructured":"De Ries, K.E., Schaap, H., van Loon, A.M., Kral, M.M., and Meijer, P.C. (2021). A literature review of open-ended concept maps as a research instrument to study knowledge and learning. Qual. Quant., 1\u201335.","DOI":"10.1007\/s11135-021-01113-x"},{"key":"ref_47","doi-asserted-by":"crossref","unstructured":"Giabbanelli, P.J. (2018). Analyzing the complexity of behavioural factors influencing weight in adults. Advanced Data Analytics in Health, Springer.","DOI":"10.1007\/978-3-319-77911-9_10"},{"key":"ref_48","first-page":"3","article-title":"When simple interpretations create complex problems","volume":"1","author":"Fink","year":"2017","journal-title":"Syst. Sci. Popul. Health"},{"key":"ref_49","doi-asserted-by":"crossref","unstructured":"Axelrod, R. (2015). Structure of Decision: The Cognitive Maps of Political Elites. Princeton University Press.","DOI":"10.1515\/9781400871957"},{"key":"ref_50","first-page":"n1","article-title":"Comparing How Different Inquiry-Based Approaches Impact Learning Outcomes","volume":"14","author":"Tawfik","year":"2020","journal-title":"Interdiscip. J. Probl. -Based Learn."},{"key":"ref_51","doi-asserted-by":"crossref","unstructured":"Tselykh, A., Vasilev, V., Tselykh, L., and Ferreira, F.A. (2020). Influence control method on directed weighted signed graphs with deterministic causality. Ann. Oper. Res., 1\u201325.","DOI":"10.1007\/s10479-020-03587-8"},{"key":"ref_52","doi-asserted-by":"crossref","unstructured":"Tselykh, A., Vasilev, V., and Tselykh, L. (2020). Assessment of influence productivity in cognitive models. Artif. Intell. Rev., 1\u201327.","DOI":"10.1007\/s10462-020-09823-8"},{"key":"ref_53","doi-asserted-by":"crossref","first-page":"277","DOI":"10.1037\/h0046049","article-title":"Structural balance: A generalization of Heider\u2019s theory","volume":"63","author":"Cartwright","year":"1956","journal-title":"Psychol. Rev."},{"key":"ref_54","doi-asserted-by":"crossref","first-page":"181","DOI":"10.1177\/001872676702000206","article-title":"Clustering and structural balance in graphs","volume":"20","author":"Davis","year":"1967","journal-title":"Hum. Relat."},{"key":"ref_55","doi-asserted-by":"crossref","unstructured":"Patel, I., Nguyen, H., Belyi, E., Getahun, Y., Abdulkareem, S., Giabbanelli, P.J., and Mago, V. (2017). Modeling information spread in polarized communities: Transitioning from legacy media to a Facebook world. SoutheastCon 2017, IEEE.","DOI":"10.1109\/SECON.2017.7925355"},{"key":"ref_56","unstructured":"Chung, F.R., and Graham, F.C. (1997). Spectral Graph Theory. American Mathematical Society."},{"key":"ref_57","doi-asserted-by":"crossref","unstructured":"Godsil, C., and Royle, G.F. (2001). Algebraic Graph Theory. Springer Science & Business Media.","DOI":"10.1007\/978-1-4613-0163-9"},{"key":"ref_58","doi-asserted-by":"crossref","first-page":"166","DOI":"10.1002\/tea.20049","article-title":"Comparison of two concept-mapping techniques: Implications for scoring, interpretation, and use","volume":"42","author":"Yin","year":"2005","journal-title":"J. Res. Sci. Teach."},{"key":"ref_59","doi-asserted-by":"crossref","first-page":"88","DOI":"10.1016\/j.ecolmodel.2011.09.011","article-title":"Modeling the integration of stakeholder knowledge in social\u2013ecological decision-making: Benefits and limitations to knowledge diversity","volume":"229","author":"Gray","year":"2012","journal-title":"Ecol. Model."},{"key":"ref_60","doi-asserted-by":"crossref","unstructured":"Papageorgiou, E., and Kontogianni, A. (2011). Using fuzzy cognitive mapping in environmental decision making and management: A methodological primer and an application. International Perspectives on Global Environmental Change, IntechOpen.","DOI":"10.5772\/29375"},{"key":"ref_61","doi-asserted-by":"crossref","first-page":"3711","DOI":"10.1016\/j.asoc.2012.02.006","article-title":"A fuzzy cognitive map of the psychosocial determinants of obesity","volume":"12","author":"Giabbanelli","year":"2012","journal-title":"Appl. Soft Comput."}],"container-title":["Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2079-8954\/9\/2\/23\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,13]],"date-time":"2025-10-13T13:58:38Z","timestamp":1760363918000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2079-8954\/9\/2\/23"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,4,5]]},"references-count":61,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2021,6]]}},"alternative-id":["systems9020023"],"URL":"https:\/\/doi.org\/10.3390\/systems9020023","relation":{},"ISSN":["2079-8954"],"issn-type":[{"value":"2079-8954","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,4,5]]}}}