{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,14]],"date-time":"2026-07-14T14:10:07Z","timestamp":1784038207194,"version":"3.55.0"},"reference-count":40,"publisher":"MDPI AG","issue":"4","license":[{"start":{"date-parts":[[2022,12,12]],"date-time":"2022-12-12T00:00:00Z","timestamp":1670803200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["BDCC"],"abstract":"<jats:p>Human as well as algorithmic searches are performed to balance exploration and exploitation. The search task in this paper is the global optimization of a 2D multimodal function, unknown to the searcher. Thus, the task presents the following features: (i) uncertainty (i.e., information about the function can be acquired only through function observations), (ii) sequentiality (i.e., the choice of the next point to observe depends on the previous ones), and (iii) limited budget (i.e., a maximum number of sequential choices allowed to the players). The data about human behavior are gathered through a gaming app whose screen represents all the possible locations the player can click on. The associated value of the unknown function is shown to the player. Experimental data are gathered from 39 subjects playing 10 different tasks each. Decisions are analyzed in a Pareto optimality setting\u2014improvement vs. uncertainty. The experimental results show that the most significant deviations from the Pareto rationality are associated with a behavior named \u201cexasperated exploration\u201d, close to random search. This behavior shows a statistically significant association with stressful situations occurring when, according to their current belief, the human feels there are no chances to improve over the best value observed so far, while the remaining budget is running out. To classify between Pareto and Not-Pareto decisions, an explainable\/interpretable Machine Learning model based on Decision Tree learning is developed. The resulting model is used to implement a synthetic human searcher\/optimizer successively compared against Bayesian Optimization. On half of the test problems, the synthetic human results as more effective and efficient.<\/jats:p>","DOI":"10.3390\/bdcc6040155","type":"journal-article","created":{"date-parts":[[2022,12,12]],"date-time":"2022-12-12T06:11:01Z","timestamp":1670825461000},"page":"155","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":6,"title":["Explaining Exploration\u2013Exploitation in Humans"],"prefix":"10.3390","volume":"6","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-1431-576X","authenticated-orcid":false,"given":"Antonio","family":"Candelieri","sequence":"first","affiliation":[{"name":"Department of Economics, Management and Statistics, University of Milano-Bicocca, 20126 Milan, Italy"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Andrea","family":"Ponti","sequence":"additional","affiliation":[{"name":"Department of Economics, Management and Statistics, University of Milano-Bicocca, 20126 Milan, Italy"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Francesco","family":"Archetti","sequence":"additional","affiliation":[{"name":"Department of Computer Science, Systems and Communication, University of Milano-Bicocca, 20126 Milan, Italy"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2022,12,12]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"2074","DOI":"10.1037\/a0038199","article-title":"Humans use directed and random exploration to solve the explore\u2013exploit dilemma","volume":"143","author":"Wilson","year":"2014","journal-title":"J. Exp. Psychol. Gen."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"49","DOI":"10.1016\/j.cobeha.2020.10.001","article-title":"Balancing exploration and exploitation with information and randomization","volume":"38","author":"Wilson","year":"2020","journal-title":"Curr. Opin. Behav. Sci."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"17771","DOI":"10.1007\/s00500-020-05398-2","article-title":"Modelling human active search in optimizing black-box functions","volume":"24","author":"Candelieri","year":"2020","journal-title":"Soft Comput."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Candelieri, A., Ponti, A., and Archetti, F. (2021). Uncertainty quantification and exploration\u2013exploitation trade-off in humans. J. Ambient. Intell. Humaniz. Comput., 1\u201334.","DOI":"10.1007\/s12652-021-03547-5"},{"key":"ref_5","unstructured":"Candelieri, A., Ponti, A., Giordani, I., and Archetti, F. (2022). On the use of Wasserstein distance in the distributional analysis of human decision making under uncertainty. Ann. Math. Artif. Intell., 1\u201322."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Frazier, P.I. (2018). Bayesian optimization. Recent Advances in Optimization and Modeling of Contemporary Problems, INFORMS.","DOI":"10.1287\/educ.2018.0188"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Archetti, F., and Candelieri, A. (2019). Bayesian Optimization and Data Science, Springer International Publishing.","DOI":"10.1007\/978-3-030-24494-1"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Candelieri, A. (2021, January 12\u201315). A gentle introduction to bayesian optimization. Proceedings of the 2021 Winter Simulation Conference (WSC), Phoenix, AZ, USA.","DOI":"10.1109\/WSC52266.2021.9715413"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"210","DOI":"10.3758\/LB.36.3.210","article-title":"Bayesian approaches to associative learning: From passive to active learning","volume":"36","author":"Kruschke","year":"2008","journal-title":"Anim. Learn. Behav."},{"key":"ref_10","unstructured":"Sun, R. (2008). Bayesian models of cognition. Cambridge Handbook of Computational Cognitive Modelling, Cambridge University Press."},{"key":"ref_11","unstructured":"Wilson, A.G., Dann, C., Lucas, C., and Xing, E.P. (2015). The human kernel. Advances in Neural Information Processing Systems, MIT Press."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"277","DOI":"10.1037\/dec0000101","article-title":"Uncertainty and exploration","volume":"6","author":"Gershman","year":"2019","journal-title":"Decision"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"933","DOI":"10.1098\/rstb.2007.2098","article-title":"Should I stay or should I go? How the human brain manages the trade-off between exploitation and exploration","volume":"362","author":"Cohen","year":"2007","journal-title":"Philos. Trans. R. Soc. B Biol. Sci."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"703","DOI":"10.1038\/s41583-019-0220-7","article-title":"Believing in dopamine","volume":"20","author":"Gershman","year":"2019","journal-title":"Nat. Rev. Neurosci."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"7","DOI":"10.1016\/j.conb.2018.11.003","article-title":"The algorithmic architecture of exploration in the human brain","volume":"55","author":"Schulz","year":"2019","journal-title":"Curr. Opin. Neurobiol."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"20130481","DOI":"10.1098\/rstb.2013.0481","article-title":"The anatomy of choice: Dopamine and decision-making","volume":"369","author":"Friston","year":"2014","journal-title":"Philos. Trans. R. Soc. B Biol. Sci."},{"key":"ref_17","unstructured":"Borji, A., and Itti, L. (2013). Bayesian optimization explains human active search. Advances in Neural Information Processing Systems, MIT Press."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"915","DOI":"10.1038\/s41562-018-0467-4","article-title":"Generalization guides human exploration in vast decision spaces","volume":"2","author":"Wu","year":"2018","journal-title":"Nat. Hum. Behav."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Tversky, A., and Kahneman, D. (1989). Rational choice and the framing of decisions. Multiple Criteria Decision-Making and Risk Analysis Using Microcomputers, Springer.","DOI":"10.1007\/978-3-642-74919-3_4"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"209","DOI":"10.1007\/s40685-019-0093-7","article-title":"Violations of dominance in decision-making","volume":"12","author":"Kourouxous","year":"2019","journal-title":"Bus. Res."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"1216","DOI":"10.1038\/s41567-019-0732-0","article-title":"The ergodicity problem in economics","volume":"15","author":"Peters","year":"2019","journal-title":"Nat. Phys."},{"key":"ref_22","unstructured":"Schumpeter, J.A. (1954). History of Economic Analysis, Psychology Press, Taylor and Francis an Informa Business."},{"key":"ref_23","unstructured":"Kahneman, D. (2011). Thinking, Fast and Slow, Farrar, Straus & Giroux."},{"key":"ref_24","unstructured":"Schulz, E., Tenenbaum, J.B., Reshef, D.N., Speekenbrink, M., and Gershman, S. (2015, January 22\u201325). Assessing the perceived predictability of functions. Proceedings of the 37th Annual Meeting of the Cognitive Science Society, CogSci 2015, Pasadena, CA, USA."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"34","DOI":"10.1016\/j.cognition.2017.12.014","article-title":"Deconstructing the human algorithms for exploration","volume":"173","author":"Gershman","year":"2018","journal-title":"Cognition"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"599","DOI":"10.1007\/s10898-018-0622-5","article-title":"Bi-objective decision making in global optimization based on statistical models","volume":"74","author":"Calvin","year":"2019","journal-title":"J. Glob. Optim."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"235","DOI":"10.1023\/A:1013689704352","article-title":"Finite-time analysis of the multiarmed bandit problem","volume":"47","author":"Auer","year":"2002","journal-title":"Mach. Learn."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"3250","DOI":"10.1109\/TIT.2011.2182033","article-title":"Information-theoretic regret bounds for gaussian process optimization in the bandit setting","volume":"58","author":"Srinivas","year":"2012","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_29","unstructured":"Iwazaki, S., Inatsu, Y., and Takeuchi, I. (2021, January 13\u201315). Mean-variance analysis in Bayesian optimization under uncertainty. Proceedings of the 24th International Conference on Artificial Intelligence and Statistics, Virtually."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"De Ath, G., Everson, R.M., Fieldsend, J.E., and Rahat, A.A. (2020, January 8\u201312). \u03f5-shotgun: \u03f5-greedy batch bayesian optimisation. Proceedings of the 2020 Genetic and Evolutionary Computation Conference, Canc\u00fan, Mexico.","DOI":"10.1145\/3377930.3390154"},{"key":"ref_31","first-page":"1","article-title":"Greed is good: Exploration and exploitation trade-offs in Bayesian optimisation","volume":"1","author":"Everson","year":"2021","journal-title":"ACM Trans. Evol. Learn. Optim."},{"key":"ref_32","first-page":"1","article-title":"Inverse Bayesian Optimization: Learning Human Acquisition Functions in an Exploration vs Exploitation Search Task","volume":"18","author":"Sandholtz","year":"2022","journal-title":"Bayesian Anal."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Gajawada, S. (2018). Artificial Human Optimization-An Introduction. Trans. Mach. Learn. Artif. Intell., 6.","DOI":"10.14738\/tmlai.64.4956"},{"key":"ref_34","first-page":"1","article-title":"Ten artificial human optimization algorithms","volume":"6","author":"Gajawada","year":"2019","journal-title":"Trans. Mach. Learn. Artif. Intell."},{"key":"ref_35","unstructured":"Brochu, E. (2010). Interactive Bayesian Optimization: Learning User Preferences for Graphics and Animation. [Ph.D. Thesis, University of British Columbia]."},{"key":"ref_36","unstructured":"Astudillo, R., and Frazier, P. (2020, January 26\u201328). Multi-attribute Bayesian optimization with interactive preference learning. Proceedings of the Twenty Third International Conference on Artificial Intelligence and Statistics, Virtually."},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Williams, C.K., and Rasmussen, C.E. (2006). Gaussian Processes for Machine Learning, MIT Press.","DOI":"10.7551\/mitpress\/3206.001.0001"},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Gramacy, R.B. (2020). Surrogates: Gaussian Process Modeling, Design, and Optimization for the Applied Sciences, CRC Press.","DOI":"10.1201\/9780367815493"},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"571","DOI":"10.1007\/s10589-020-00215-w","article-title":"Global optimization via inverse distance weighting and radial basis functions","volume":"77","author":"Bemporad","year":"2020","journal-title":"Comput. Optim. Appl."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"10","DOI":"10.1145\/1656274.1656278","article-title":"The WEKA data mining software: An update","volume":"11","author":"Hall","year":"2009","journal-title":"ACM SIGKDD Explor. Newsl."}],"container-title":["Big Data and Cognitive Computing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2504-2289\/6\/4\/155\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T01:39:48Z","timestamp":1760146788000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2504-2289\/6\/4\/155"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,12,12]]},"references-count":40,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2022,12]]}},"alternative-id":["bdcc6040155"],"URL":"https:\/\/doi.org\/10.3390\/bdcc6040155","relation":{},"ISSN":["2504-2289"],"issn-type":[{"value":"2504-2289","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,12,12]]}}}