{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,3]],"date-time":"2026-07-03T19:42:37Z","timestamp":1783107757901,"version":"3.54.6"},"reference-count":38,"publisher":"Association for Computing Machinery (ACM)","issue":"2","license":[{"start":{"date-parts":[[2024,6,24]],"date-time":"2024-06-24T00:00:00Z","timestamp":1719187200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Research Council of Finland (flagship program: Finnish Center for Artificial Intelligence","award":["328400, 345604, 341763"],"award-info":[{"award-number":["328400, 345604, 341763"]}]},{"name":"subjective functions","award":["357578"],"award-info":[{"award-number":["357578"]}]},{"DOI":"10.13039\/100020595","name":"National Science and Technology Council of Taiwan","doi-asserted-by":"crossref","award":["NSTC 112-2221-E-A49-125"],"award-info":[{"award-number":["NSTC 112-2221-E-A49-125"]}],"id":[{"id":"10.13039\/100020595","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/501100000266","name":"EPSRC","doi-asserted-by":"crossref","award":["EP\/S027432\/1 and EP\/W02456X\/1"],"award-info":[{"award-number":["EP\/S027432\/1 and EP\/W02456X\/1"]}],"id":[{"id":"10.13039\/501100000266","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Interact. Intell. Syst."],"published-print":{"date-parts":[[2024,6,30]]},"abstract":"<jats:p>Computational methods can potentially facilitate user interface design by complementing designer intuition, prior experience, and personal preference. Framing a user interface design task as a multi-objective optimization problem can help with operationalizing and structuring this process at the expense of designer agency and experience. While offering a systematic means of exploring the design space, the optimization process cannot typically leverage the designer\u2019s expertise in quickly identifying that a given \u201cbad\u201d design is not worth evaluating. We here examine a cooperative approach where both the designer and optimization process share a common goal and work in partnership by establishing a shared understanding of the design space. We tackle the research question: How can we foster cooperation between the designer and a systematic optimization process in order to best leverage their combined strength? We introduce and present an evaluation of a cooperative approach that allows the user to express their design insight and work in concert with a multi-objective design process. We find that the cooperative approach successfully encourages designers to explore more widely in the design space than when they are working without assistance from an optimization process. The cooperative approach also delivers design outcomes that are comparable to an optimization process run without any direct designer input but achieves this with greater efficiency and substantially higher designer engagement levels.<\/jats:p>","DOI":"10.1145\/3657643","type":"journal-article","created":{"date-parts":[[2024,4,17]],"date-time":"2024-04-17T14:57:54Z","timestamp":1713365874000},"page":"1-28","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":11,"title":["Cooperative Multi-Objective Bayesian Design Optimization"],"prefix":"10.1145","volume":"14","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-9535-778X","authenticated-orcid":false,"given":"George","family":"Mo","sequence":"first","affiliation":[{"name":"University of Cambridge, Cambridge, United Kingdom"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6692-4853","authenticated-orcid":false,"given":"John","family":"Dudley","sequence":"additional","affiliation":[{"name":"University of Cambridge, Cambridge, United Kingdom"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7862-2817","authenticated-orcid":false,"given":"Liwei","family":"Chan","sequence":"additional","affiliation":[{"name":"National Yang Ming Chiao Tung University, Hsinchu, Taiwan"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2670-8328","authenticated-orcid":false,"given":"Yi-Chi","family":"Liao","sequence":"additional","affiliation":[{"name":"Aalto University, Aalto, Finland"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2498-7837","authenticated-orcid":false,"given":"Antti","family":"Oulasvirta","sequence":"additional","affiliation":[{"name":"Aalto University, Aalto, Finland"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7139-871X","authenticated-orcid":false,"given":"Per Ola","family":"Kristensson","sequence":"additional","affiliation":[{"name":"University of Cambridge, Cambridge, United Kingdom"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2024,6,24]]},"reference":[{"key":"e_1_3_3_2_2","doi-asserted-by":"publisher","DOI":"10.1145\/2501988.2502024"},{"key":"e_1_3_3_3_2","first-page":"21524","article-title":"BoTorch: A framework for efficient Monte-Carlo Bayesian optimization","volume":"33","author":"Balandat Maximilian","year":"2020","unstructured":"Maximilian Balandat, Brian Karrer, Daniel R. Jiang, Samuel Daulton, Benjamin Letham, Andrew Gordon Wilson, and Eytan Bakshy. 2020. BoTorch: A framework for efficient Monte-Carlo Bayesian optimization. Advances in Neural Information Processing Systems 33 (2020), 21524\u201321538. Retrieved from http:\/\/arxiv.org\/abs\/1910.06403","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_3_4_2","doi-asserted-by":"publisher","DOI":"10.5555\/2835587.2835589"},{"key":"e_1_3_3_5_2","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2013.61"},{"key":"e_1_3_3_6_2","first-page":"1","volume-title":"Advances in Neural Information Processing Systems.","author":"Borji Ali","year":"2013","unstructured":"Ali Borji and Laurent Itti. 2013. Bayesian optimization explains human active search. In Advances in Neural Information Processing Systems. C. J. Burges, L. Bottou, M. Welling, Z. Ghahramani, and K. Q. Weinberger (Eds.), Vol. 26. Curran Associates, Inc. 1\u20139. Retrieved from https:\/\/proceedings.neurips.cc\/paper\/2013\/file\/a3f390d88e4c41f2747bfa2f1b5f87db-Paper.pdf"},{"key":"e_1_3_3_7_2","first-page":"103","volume-title":"Proceedings of the ACM SIGGRAPH\/Eurographics Symposium on Computer Animation (SCA \u201910)","author":"Brochu Eric","year":"2010","unstructured":"Eric Brochu, Tyson Brochu, and Nando de Freitas. 2010. A Bayesian interactive optimization approach to procedural animation design. In Proceedings of the ACM SIGGRAPH\/Eurographics Symposium on Computer Animation (SCA \u201910). Eurographics Association, Goslar, DEU, 103\u2013112."},{"key":"e_1_3_3_8_2","doi-asserted-by":"publisher","DOI":"10.1145\/3491102.3501850"},{"key":"e_1_3_3_9_2","doi-asserted-by":"publisher","DOI":"10.1145\/3173574.3174070"},{"key":"e_1_3_3_10_2","doi-asserted-by":"publisher","DOI":"10.1145\/2617588"},{"key":"e_1_3_3_11_2","doi-asserted-by":"publisher","DOI":"10.1038\/d41586-021-01170-0"},{"key":"e_1_3_3_12_2","doi-asserted-by":"publisher","unstructured":"Allan Dafoe Edward Hughes Yoram Bachrach Tantum Collins Kevin R. McKee Joel Z. Leibo Kate Larson and Thore Graepel. 2020. Open problems in cooperative AI. arXiv:2012.08630 [cs]. Retrieved from 10.48550\/arXiv.2012.08630","DOI":"10.48550\/arXiv.2012.08630"},{"key":"e_1_3_3_13_2","doi-asserted-by":"publisher","DOI":"10.5555\/3495724.3496550"},{"key":"e_1_3_3_14_2","volume-title":"Human-Computer Interaction","author":"Dix Alan","year":"2003","unstructured":"Alan Dix, Janet Finlay, Gregory D. Abowd, and Russel Beale. 2003. Human-Computer Interaction. Pearson\/Prentice-Hall. Retrieved from https:\/\/books.google.co.uk\/books?id=IuQxui8GHDcC"},{"key":"e_1_3_3_15_2","doi-asserted-by":"publisher","DOI":"10.1016\/S0142-694X(01)00009-6"},{"key":"e_1_3_3_16_2","first-page":"259","article-title":"Bayesian optimisation of interface features","author":"Dudley John","year":"2022","unstructured":"John Dudley and Per O. Kristensson. 2022. Bayesian optimisation of interface features. In Bayesian Methods for Interaction and Design. John H. Williamson, Antti Oulasvirta, Per Ola Kristensson, and Nikola Banovic (Eds.), Cambridge University Press, 259\u2013284.","journal-title":"Bayesian Methods for Interaction and Design"},{"key":"e_1_3_3_17_2","doi-asserted-by":"publisher","DOI":"10.1145\/3290605.3300482"},{"key":"e_1_3_3_18_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.softx.2020.100520"},{"key":"e_1_3_3_19_2","doi-asserted-by":"publisher","DOI":"10.1145\/3411764.3445140"},{"key":"e_1_3_3_20_2","doi-asserted-by":"publisher","DOI":"10.1145\/1753326.1753529"},{"key":"e_1_3_3_21_2","doi-asserted-by":"publisher","DOI":"10.1145\/2858036.2858253"},{"key":"e_1_3_3_22_2","unstructured":"Nicolas Knudde Joachim van der Herten Tom Dhaene and Ivo Couckuyt. 2017. GPflowOpt: A Bayesian optimization library using TensorFlow. arXiv:1711.03845. Retrieved from https:\/\/arxiv.org\/abs\/1711.03845"},{"key":"e_1_3_3_23_2","doi-asserted-by":"publisher","DOI":"10.1145\/3386569.3392444"},{"key":"e_1_3_3_24_2","doi-asserted-by":"publisher","DOI":"10.1145\/3072959.3073598"},{"key":"e_1_3_3_25_2","doi-asserted-by":"publisher","DOI":"10.1109\/MPRV.2022.3230597"},{"key":"e_1_3_3_26_2","doi-asserted-by":"publisher","DOI":"10.1145\/2858036.2858425"},{"key":"e_1_3_3_27_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10994-020-05899-z"},{"key":"e_1_3_3_28_2","doi-asserted-by":"publisher","DOI":"10.1145\/97243.97281"},{"key":"e_1_3_3_29_2","doi-asserted-by":"publisher","DOI":"10.1145\/2702123.2702149"},{"key":"e_1_3_3_30_2","unstructured":"OED Online. 2022. cooperative adj. and n. Retrieved from https:\/\/www.oed.com\/view\/Entry\/41038"},{"key":"e_1_3_3_31_2","doi-asserted-by":"publisher","DOI":"10.1145\/3581641.3584066"},{"key":"e_1_3_3_32_2","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2014.2346321"},{"key":"e_1_3_3_33_2","first-page":"1919","volume-title":"Proceedings of the 33rd International Conference on Machine Learning (ICML \u201916)","author":"Shah Amar","year":"2016","unstructured":"Amar Shah and Zoubin Ghahramani. 2016. Pareto Frontier learning with expensive correlated objectives. In Proceedings of the 33rd International Conference on Machine Learning (ICML \u201916). PMLR, 1919\u20131927. Retrieved from http:\/\/proceedings.mlr.press\/v48\/shahc16.html"},{"key":"e_1_3_3_34_2","doi-asserted-by":"publisher","DOI":"10.1109\/JPROC.2015.2494218"},{"key":"e_1_3_3_35_2","doi-asserted-by":"publisher","DOI":"10.1109\/INFVIS.1995.528680"},{"key":"e_1_3_3_36_2","doi-asserted-by":"publisher","DOI":"10.1145\/2901790.2901817"},{"key":"e_1_3_3_37_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.iswa.2022.200151"},{"key":"e_1_3_3_38_2","doi-asserted-by":"publisher","DOI":"10.1111\/cgf.13415"},{"key":"e_1_3_3_39_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.future.2022.05.014"}],"container-title":["ACM Transactions on Interactive Intelligent Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3657643","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3657643","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,19]],"date-time":"2025-06-19T00:05:53Z","timestamp":1750291553000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3657643"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,6,24]]},"references-count":38,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2024,6,30]]}},"alternative-id":["10.1145\/3657643"],"URL":"https:\/\/doi.org\/10.1145\/3657643","relation":{},"ISSN":["2160-6455","2160-6463"],"issn-type":[{"value":"2160-6455","type":"print"},{"value":"2160-6463","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,6,24]]},"assertion":[{"value":"2023-04-05","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-01-02","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-06-24","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}