{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,12]],"date-time":"2025-10-12T04:17:43Z","timestamp":1760242663587,"version":"build-2065373602"},"reference-count":83,"publisher":"MDPI AG","issue":"1","license":[{"start":{"date-parts":[[2016,1,15]],"date-time":"2016-01-15T00:00:00Z","timestamp":1452816000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Systems"],"abstract":"<jats:p>Imagine a person visiting an urban event. At each moment in time, the person has to weigh up different possible actions and make consecutive decisions. For instance, a person might be hungry or thirsty and would therefore like to go somewhere to eat or to drink, or a person might need to go to the toilet and thus go searching for the restrooms. Other possible desires might be to go dancing or to have a rest due to exhaustion. All these examples can be seen in the context of dynamic decision-making. To be able to implement the dynamic decision-making of virtual humans living their lives in a persistent microworld, an advanced concept to solve this\u2014in artificial intelligence research commonly called action selection problem\u2014is required. This article focuses on an novel approach to model the activation of motivations\u2014as an attempt to answer the recurring question of the virtual humans \u201cWhat to do next?\u201d. The novelty is to use System Dynamics, in general defined as a top-down simulation approach, from the bottom-up inside each instance of the agent population and to implement an action selection mechanism on the basis of this methodology. This approach enables us to model the dynamic decision-making of the virtual humans with stocks and flows resulting in nonlinear motivation evolution. A case study in the context of an urban event documents the application of this innovative method.<\/jats:p>","DOI":"10.3390\/systems4010004","type":"journal-article","created":{"date-parts":[[2016,1,16]],"date-time":"2016-01-16T03:30:25Z","timestamp":1452915025000},"page":"4","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":5,"title":["Modeling Dynamic Decision-Making of Virtual Humans"],"prefix":"10.3390","volume":"4","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-7493-3303","authenticated-orcid":false,"given":"Oliver","family":"Handel","sequence":"first","affiliation":[{"name":"Department of Civil, Geo and Environmental Engineering, Technical University Munich, Arcisstr. 21, Munich D-80290, Germany"}]}],"member":"1968","published-online":{"date-parts":[[2016,1,15]]},"reference":[{"key":"ref_1","unstructured":"Busemeyer, J.R. (1999). International Encyclopedia of the Social and Behavioral Sciences: Methodology, Mathematics and Computer Science, Elsevier."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"211","DOI":"10.1016\/0001-6918(92)90019-A","article-title":"Dynamic decision making: Human control of complex systems","volume":"81","author":"Brehmer","year":"1992","journal-title":"Acta Psychol."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"60","DOI":"10.3390\/systems3020060","article-title":"Introduction to the Special Issue Dynamic Decision-Making in Controlled Experiments","volume":"3","author":"Stouten","year":"2015","journal-title":"Systems"},{"key":"ref_4","first-page":"591","article-title":"Instance-based learning in dynamic decision making","volume":"27","author":"Gonzalez","year":"2003","journal-title":"Cogn. Sci."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Borshchev, A. (2013). The Big Book of Simulation Modeling: Multimethod Modeling with AnyLogic 6, AnyLogic North American.","DOI":"10.1002\/9781118762745.ch12"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"71","DOI":"10.1145\/356835.356839","article-title":"Human-computer interaction in the control of dynamic systems","volume":"13","author":"Rouse","year":"1981","journal-title":"ACM Comput. Surv. (CSUR)"},{"key":"ref_7","unstructured":"Sterman, J. (2000). Business Dynamics. Systems Thinking and Modeling for a Complex World, Irwin\/McGraw-Hill."},{"key":"ref_8","unstructured":"Ulrich, H., and Probst, G. (1995). Anleitung zum Ganzheitlichen Denken und Handeln: Ein Brevier f\u00fcr F\u00fchrungskr\u00e4fte, Pail Haupt."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"8","DOI":"10.2307\/41166098","article-title":"System dynamics modeling","volume":"43","author":"Sterman","year":"2001","journal-title":"Calif. Manag. Rev."},{"key":"ref_10","first-page":"5520","article-title":"Model developement for integrated assessment","volume":"40","author":"Letcher","year":"2004","journal-title":"Water Resour."},{"key":"ref_11","unstructured":"Janssen, M.A. (2005). Internet Encyclopaedia of Ecological Econom ics, Arizona State University."},{"key":"ref_12","unstructured":"Funke, J. (1995). Complex Problem Solving: The European Perspective, Psychology Press."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"273","DOI":"10.1016\/j.chb.2004.02.014","article-title":"The use of microworlds to study dynamic decision making","volume":"21","author":"Gonzalez","year":"2005","journal-title":"Comput. Hum. Behav."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"331","DOI":"10.1002\/bdm.381","article-title":"Taking stock of naturalistic decision making","volume":"14","author":"Lipshitz","year":"2001","journal-title":"J. Behav. Decis. Mak."},{"key":"ref_15","unstructured":"Schelling, T. (2006). Micromotives and Macrobehavior, WW Norton & Company."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"916","DOI":"10.1137\/S0036139997332099","article-title":"Resurrection of \u201csecond order\u201d models of traffic flow","volume":"60","author":"Aw","year":"2000","journal-title":"SIAM J. Appl. Math."},{"key":"ref_17","unstructured":"Goodman, M., Kreutzer, W.B., Sterman, J., and Kreutzer, D.P. (1993). \u201cElectrifying\u201d Learning: Computerizing the Beer Game, Prentice Hall."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"301","DOI":"10.1016\/0749-5978(89)90041-1","article-title":"Misperceptions of feedback in dynamic decision making","volume":"43","author":"Sterman","year":"1989","journal-title":"Organ. Behav. Hum. Decis. Process."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"164","DOI":"10.1002\/bs.3830070203","article-title":"The design of a fungus\u2014Eater: A model of human behavior in an unsophisticated environment","volume":"7","author":"Toda","year":"1962","journal-title":"Behav. Sci."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"49","DOI":"10.5751\/ES-01741-110149","article-title":"Agent-based analysis of agricultural policies: An illustration of the agricultural policy simulator AgriPoliS, its adaptation and behavior","volume":"11","author":"Happe","year":"2006","journal-title":"Ecol. Soc."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"180","DOI":"10.1038\/nature02541","article-title":"Modelling disease outbreaks in realistic urban social networks","volume":"429","author":"Eubank","year":"2004","journal-title":"Nature"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"255","DOI":"10.1016\/j.trc.2005.07.002","article-title":"An analysis of agent-based approaches to transport logistics","volume":"13","author":"Davidsson","year":"2005","journal-title":"Transp. Res. Part C: Emerg. Technol."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"25","DOI":"10.1057\/palgrave.jors.2602321","article-title":"Agent-based modelling and simulation of urban evacuation: Relative effectiveness of simultaneous and staged evacuation strategies","volume":"59","author":"Chen","year":"2008","journal-title":"J. Oper. Res. Soc."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"359","DOI":"10.1111\/j.1467-9450.2007.00596.x","article-title":"Achieving what cannot be done: Coping with the time constants in a dynamic decision task by doing something else","volume":"48","author":"Brehmer","year":"2007","journal-title":"Scand. J. Psychol."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"119","DOI":"10.1002\/sdr.267","article-title":"Understanding and control of a simple dynamic system","volume":"19","author":"Jensen","year":"2003","journal-title":"Syst. Dyn. Rev."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"89","DOI":"10.1016\/0001-6918(94)90013-2","article-title":"The effect of time pressure on decision-making behaviour in a dynamic task environment","volume":"86","author":"Kerstholt","year":"1994","journal-title":"Acta Psychol."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"1904","DOI":"10.1016\/j.chb.2011.04.015","article-title":"A generic dynamic control task for behavioral research and education","volume":"27","author":"Gonzalez","year":"2011","journal-title":"Comput. Hum. Behav."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"929","DOI":"10.1109\/21.247880","article-title":"Supervisory control in a dynamic and uncertain environment: A process model of skilled human-environment interaction","volume":"23","author":"Kirlik","year":"1993","journal-title":"Syst. Man Cyber."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"113","DOI":"10.1007\/s00146-007-0126-1","article-title":"A multi-agent based framework for the simulation of human and social behaviors during emergency evacuations","volume":"22","author":"Pan","year":"2007","journal-title":"Ai & Soc."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"669","DOI":"10.1016\/j.trpro.2014.09.072","article-title":"A System Dynamics based Perspective to Help to Understand the Managerial Big Picture in Respect of Urban Event Dynamics","volume":"2","author":"Handel","year":"2014","journal-title":"Transp. Res. Procedia"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"7","DOI":"10.1140\/epjds7","article-title":"Crowd disasters as systemic failures: Analysis of the Love Parade disaster","volume":"1","author":"Helbing","year":"2012","journal-title":"EPJ Data Sci."},{"key":"ref_32","unstructured":"Sevin, E., Thalmann, D., De Sevin, E., and Thalmann, D. (2005). Computer Graphics International 2005, IEEE. number VRLAB-CONF-2005-014."},{"key":"ref_33","unstructured":"Tyrell, T. (1994). Defining the Action Selection Problem, Centre for Cognitive Science."},{"key":"ref_34","unstructured":"Tyrrell, T. (1993). Computational Mechanisms for Action Selection, University of Edinburgh."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"169","DOI":"10.1016\/S0191-2615(03)00007-9","article-title":"Pedestrian route-choice and activity scheduling theory and models","volume":"38","author":"Hoogendoorn","year":"2004","journal-title":"Transp. Res. Part B Methodol."},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Blumberg, B., and Galyean, T. (1995, January 27\u201329). Multi-level direction of autonomous creatures for real-time virtual environments. Proceedings of the 22nd Annual Conference on Computer Graphics and Interactive Techniques, New York, NY, USA.","DOI":"10.1145\/218380.218405"},{"key":"ref_37","unstructured":"Reynolds, C.W. Steering behaviors for autonomous characters. Proceedings of the Game Developers Conference."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"263","DOI":"10.1002\/wcs.1223","article-title":"Navigation and steering for autonomous virtual humans","volume":"4","author":"Kapadia","year":"2013","journal-title":"Wiley Interdiscip. Rev. Cogn. Sci."},{"key":"ref_39","first-page":"2221","article-title":"A cellular automaton model for freeway traffic","volume":"2","author":"Nagel","year":"1992","journal-title":"J. Phys."},{"key":"ref_40","unstructured":"Blue, V.V., Embrechts, M., and Adler, J.J. (1997, January 12\u20135). Cellular automata modeling of pedestrian movements. Proceedings of the IEEE International Conference on Systems, Man and Cybernetics, Orlando, FL, USA."},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"4282","DOI":"10.1103\/PhysRevE.51.4282","article-title":"Social force model for pedestrian dynamics","volume":"51","author":"Helbing","year":"1995","journal-title":"Phys. Rev. E"},{"key":"ref_42","unstructured":"Kneidl, A., and Borrmann, A. (2013). Methoden zur Abbildung Menschlichen Navigationsverhaltens Bei der Modellierung von Fu\u00dfg\u00e4ngerstr\u00f6men. [Ph.D. Thesis, Technische Universit\u00e4t M\u00fcnchen]."},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Helbing, D. (1995). Quantitative Sociodynamics, Springer.","DOI":"10.1007\/978-94-015-8516-3"},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"139","DOI":"10.1007\/BF00317988","article-title":"Simulation of chaotic EEG patterns with a dynamic model of the olfactory system","volume":"56","author":"Freeman","year":"1987","journal-title":"Biol. Cybern."},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"451","DOI":"10.1142\/S0218127492000653","article-title":"Tutorial on neurobiology: From single neurons to brain chaos","volume":"2","author":"Freeman","year":"1992","journal-title":"Int. J. Bifurc. Chaos"},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"404","DOI":"10.1177\/014616727600200413","article-title":"\u201cIn General\u201d vs. \u201cIt Depends\u201d: Some Comments of the Gergen-Schlenker Debate","volume":"2","author":"Thorngate","year":"1976","journal-title":"Pers. Soc. Psychol. Bull."},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"373","DOI":"10.1177\/014616727600200409","article-title":"Social psychology, science and history","volume":"2","author":"Gergen","year":"1976","journal-title":"Pers. Soc. Psychol. Bull."},{"key":"ref_48","first-page":"797","article-title":"Theory construction as disciplined reflexivity: Tradeoffs in the 90s","volume":"24","author":"Weick","year":"1999","journal-title":"Acad. Manag. Rev."},{"key":"ref_49","doi-asserted-by":"crossref","first-page":"321","DOI":"10.1287\/mnsc.35.3.321","article-title":"Modeling managerial behavior: Misperceptions of feedback in a dynamic decision making experiment","volume":"35","author":"Sterman","year":"1989","journal-title":"Manag. Sci."},{"key":"ref_50","unstructured":"Simon, H. (1982). Models of Bounded Rationality: Empirically Grounded Economic Reason, MIT Press."},{"key":"ref_51","first-page":"596","article-title":"Preference reversals: A broader perspective","volume":"73","author":"Slovic","year":"1983","journal-title":"Am. Econ. Rev."},{"key":"ref_52","doi-asserted-by":"crossref","first-page":"680","DOI":"10.1287\/mnsc.31.6.680","article-title":"Cognitive heuristics and feedback in a dynamic decision environment","volume":"31","author":"Kleinmuntz","year":"1985","journal-title":"Manag. Sci."},{"key":"ref_53","doi-asserted-by":"crossref","unstructured":"Miller, J., and Page, S. (2009). Complex Adaptive Systems: An Introduction to Computational Models of Social Life: An Introduction to Computational Models of Social Life, Princeton University Press.","DOI":"10.1515\/9781400835522"},{"key":"ref_54","unstructured":"Russell, S., Norvig, P., Canny, J., Malik, J., and Edwards, D. (1995). Artificial Intelligence: A Modern Approach, Prentice Hall."},{"key":"ref_55","unstructured":"Luck, M., D\u2019Inverno, M., and Munroe, S. (2003). Agent Autonomy, Springer."},{"key":"ref_56","first-page":"148","article-title":"Designing emotions for activity selection in autonomous agents","volume":"115","year":"2003","journal-title":"Emot. Hum. Artifacts"},{"key":"ref_57","doi-asserted-by":"crossref","unstructured":"Balkenius, C. (1993, January 25\u201329). The roots of motivation. Proceedings of the Second International Conference on From Animals to Animats, Rome, Italy.","DOI":"10.7551\/mitpress\/3116.003.0070"},{"key":"ref_58","doi-asserted-by":"crossref","first-page":"217","DOI":"10.1080\/02699938708408049","article-title":"Motives, mechanisms, and emotions","volume":"1","author":"Sloman","year":"1987","journal-title":"Cogn. Emot."},{"key":"ref_59","unstructured":"Sloman, A., and Croucher, M. Why Robots Will Have Emotions. Available online: http:\/\/cogprints.org\/705\/."},{"key":"ref_60","first-page":"435","article-title":"Intelligent agents in computer games","volume":"1999","author":"Lent","year":"1999","journal-title":"AAAI\/IAAI"},{"key":"ref_61","unstructured":"Senge, P.M. (2006). The Fifth Discipline: The Art and Practice of the Learning Organization, Random House LLC. [1st ed.]."},{"key":"ref_62","unstructured":"Ford, F.A. (1999). Modeling the Environment: An Introduction to System Dynamics Models of Environmental Systems, Island Press."},{"key":"ref_63","doi-asserted-by":"crossref","first-page":"432","DOI":"10.1037\/0033-295X.100.3.432","article-title":"Decision field theory: A dynamic-cognitive approach to decision making in an uncertain environment","volume":"100","author":"Busemeyer","year":"1993","journal-title":"Psychol. Rev."},{"key":"ref_64","doi-asserted-by":"crossref","first-page":"310","DOI":"10.1006\/jmps.1998.1249","article-title":"A Stochastic Version of General Recognition Theory","volume":"44","author":"Ashby","year":"2000","journal-title":"J. Math. Psychol."},{"key":"ref_65","unstructured":"Kiyosi, I. (2010). Sample Paths for a Diffusion Process, Wiley Online Library."},{"key":"ref_66","first-page":"144","article-title":"A hierarchical classifier system implementing a motivationally autonomous animat","volume":"3","author":"Donnart","year":"1994","journal-title":"Anim. Animat."},{"key":"ref_67","doi-asserted-by":"crossref","first-page":"387","DOI":"10.1177\/105971239300100401","article-title":"The use of hierarchies for action selection","volume":"1","author":"Tyrrell","year":"1993","journal-title":"Adapt. Behav."},{"key":"ref_68","unstructured":"Schmidt, B. (2002). Modelling of Human Behaviour The PECS Reference Model, SCS Publications."},{"key":"ref_69","first-page":"285","article-title":"Human performance simulation","volume":"2003","author":"Silverman","year":"2003","journal-title":"Methods Hum Perform."},{"key":"ref_70","doi-asserted-by":"crossref","first-page":"163","DOI":"10.1162\/pres.2006.15.2.163","article-title":"Human behavior models for agents in simulators and games: Part II: Gamebot engineering with PMFserv","volume":"15","author":"Silverman","year":"2006","journal-title":"Teleoper. Virtual Environ."},{"key":"ref_71","doi-asserted-by":"crossref","first-page":"139","DOI":"10.1162\/pres.2006.15.2.139","article-title":"Human behavior models for agents in simulators and games: Part I: enabling science with PMFserv","volume":"15","author":"Silverman","year":"2006","journal-title":"Teleoper. Virtual Environ."},{"key":"ref_72","doi-asserted-by":"crossref","unstructured":"Magnenat-Thalmann, N., and Thalmann, D. (2005). Handbook of Virtual Humans, Wiley Online Library.","DOI":"10.1002\/0470023198"},{"key":"ref_73","unstructured":"Gr\u00f6\u00dfler, A., and Schieritz, N. (2005). Research Methodologiesin Supply Chain Management, Physica-VerlagHD."},{"key":"ref_74","doi-asserted-by":"crossref","first-page":"1691","DOI":"10.1016\/0006-2952(76)90400-7","article-title":"Control of brain neurotransmitter synthesis by precursor availability and nutritional state","volume":"25","author":"Wurtman","year":"1976","journal-title":"Biochem. Pharmacol."},{"key":"ref_75","first-page":"1","article-title":"Neurotransmitters","volume":"5","author":"Deutch","year":"1999","journal-title":"Fundam. Neurosci."},{"key":"ref_76","doi-asserted-by":"crossref","unstructured":"Perry, E., Ashton, H., and Young, A. (2002). Neurochemistry of Consciousness: Neurotransmitters in Mind, John Benjamins Publishing.","DOI":"10.1075\/aicr.36"},{"key":"ref_77","unstructured":"Kadle\u010dek, D., and Nahodil, P. (2001). Advance in Artificial Life, Springer."},{"key":"ref_78","unstructured":"Moxnes, E., and Alessi, S. (2014, January 20\u201324). Testing training programs to limit binge drinking. Proceedings of the 32nd International Conference of the System Dynamics Society, Delft, The Netherlands."},{"key":"ref_79","doi-asserted-by":"crossref","first-page":"63","DOI":"10.1016\/j.drugalcdep.2009.06.012","article-title":"Drunker than intended: Misperceptions and information treatments","volume":"105","author":"Moxnes","year":"2009","journal-title":"Drug Alcohol Depend."},{"key":"ref_80","unstructured":"Biedermann, D., Dietrich, F., Handel, O., Kielar, P.M., and Seitz, M. (2015). Using Raspberry Pi for Scientific Video Observation of Pedestrians During a Music Festival, Technische Universit\u00e4t M\u00fcnchen, Intitut f\u00fcr Informatik. Technical Report."},{"key":"ref_81","first-page":"1","article-title":"Dynamic Visualization of Pedestrian Simulation Data","volume":"2015","author":"Handel","year":"2015","journal-title":"Forum Bauinf."},{"key":"ref_82","unstructured":"B\u00fcchele, D. (2014). Visualisierung von Fu\u00dfg\u00e4ngersimulationsdaten auf Basis Einer 3D-Game-Engine. [Ph.D. Thesis, Technische Universit\u00e4t M\u00fcnchen]."},{"key":"ref_83","first-page":"78","article-title":"Leverage Points: Places to Intervene in a System","volume":"91","author":"Meadows","year":"1999","journal-title":"Solut. Sustain. Desir. Future"}],"container-title":["Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2079-8954\/4\/1\/4\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T19:17:45Z","timestamp":1760210265000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2079-8954\/4\/1\/4"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2016,1,15]]},"references-count":83,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2016,3]]}},"alternative-id":["systems4010004"],"URL":"https:\/\/doi.org\/10.3390\/systems4010004","relation":{},"ISSN":["2079-8954"],"issn-type":[{"type":"electronic","value":"2079-8954"}],"subject":[],"published":{"date-parts":[[2016,1,15]]}}}