{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,7,11]],"date-time":"2025-07-11T10:18:39Z","timestamp":1752229119446,"version":"3.41.0"},"reference-count":49,"publisher":"Association for Computing Machinery (ACM)","issue":"EICS","license":[{"start":{"date-parts":[[2019,6,13]],"date-time":"2019-06-13T00:00:00Z","timestamp":1560384000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"CNCS-UEFISCDI","award":["PN-III-P1-1.1-TE-2016-2173 (TE141\/2018)"],"award-info":[{"award-number":["PN-III-P1-1.1-TE-2016-2173 (TE141\/2018)"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Proc. ACM Hum.-Comput. Interact."],"published-print":{"date-parts":[[2019,6,13]]},"abstract":"<jats:p>We introduce AB4Web, a web-based engine that implements a balanced randomized version of the multivariate A\/B testing, specifically designed for practitioners to readily compare end-users' preferences for user interface alternatives, such as menu layouts, widgets, controls, forms, or visual input commands. AB4Web automatically generates a balanced set of randomized pairs from a pool of user interface design alternatives, presents them to participants, collects their preferences, and reports results from the perspective of four quantitative measures: the number of presentations, the preference percentage, the latent score of preference, and the matrix of preferences. In this paper, we exemplify the AB4Web tester with a user study for which N=108 participants expressed their preferences regarding the visual design of 49 distinct graphical adaptive menus, with a total number of 5,400 preference votes. We compare the results obtained from our quantitative measures with four alternative methods: Condorcet, de Borda count starting at one and zero, and the Dowdall scoring system. We plan to release AB4Web as a public tool for practitioners to create their own A\/B testing experiments.<\/jats:p>","DOI":"10.1145\/3331160","type":"journal-article","created":{"date-parts":[[2019,6,14]],"date-time":"2019-06-14T12:36:28Z","timestamp":1560515788000},"page":"1-28","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":13,"title":["AB4Web"],"prefix":"10.1145","volume":"3","author":[{"given":"Jean","family":"Vanderdonckt","sequence":"first","affiliation":[{"name":"Universit\u00e9 catholique de Louvain, Louvain-la-Neuve, Belgium"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Mathieu","family":"Zen","sequence":"additional","affiliation":[{"name":"Universit\u00e9 catholique de Louvain, Louvain-la-Neuve, Belgium"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Radu-Daniel","family":"Vatavu","sequence":"additional","affiliation":[{"name":"University \u015etefan cel Mare of Suceava, Suceava, Romania"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2019,6,13]]},"reference":[{"key":"e_1_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1145\/1753326.1753534"},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1145\/1978942.1979131"},{"key":"e_1_2_1_3_1","volume-title":"Filed June 17th, 1998","author":"Arcuri Michael P.","year":"2000","unstructured":"Michael P. Arcuri , Thomas Scott Coon , Jeffrey J. Johnson , Alexis Warren , Jacob Manning , and Martijn Eldert van Tilburg . 2000 . Adaptive Menus, Patent US6121968A, Microsoft. (Sept. 2000) . Filed June 17th, 1998 , Issued Sep. 19th. , 2000. Michael P. Arcuri, Thomas Scott Coon, Jeffrey J. Johnson, Alexis Warren, Jacob Manning, and Martijn Eldert van Tilburg. 2000. Adaptive Menus, Patent US6121968A, Microsoft. (Sept. 2000). Filed June 17th, 1998, Issued Sep. 19th., 2000."},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-02958-0_35"},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1145\/1385569.1385575"},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1145\/3002171"},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1145\/1166253.1166280"},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1145\/354401.354782"},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1145\/634067.634171"},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1037\/e576962012-008"},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1145\/2933242.2935874"},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1145\/2933242.2935874"},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1145\/3099585"},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1145\/2686612.2686619"},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1145\/2838739.2838749"},{"key":"e_1_2_1_16_1","first-page":"324","article-title":"Rank Analysis of Incomplete Block Designs","volume":"39","author":"Bradley Ralph Allan","year":"1952","unstructured":"Ralph Allan Bradley and Milton E. Terry . 1952 . Rank Analysis of Incomplete Block Designs : I. The Method of Paired Comparisons. Biometrika 39 , 3\/4 (1952), 324 -- 345 . http:\/\/www.jstor.org\/stable\/2334029 Ralph Allan Bradley and Milton E. Terry. 1952. Rank Analysis of Incomplete Block Designs: I. The Method of Paired Comparisons. Biometrika 39, 3\/4 (1952), 324--345. http:\/\/www.jstor.org\/stable\/2334029","journal-title":"I. The Method of Paired Comparisons. Biometrika"},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1145\/1111449.1111526"},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1145\/1240624.1240723"},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0950-3293(97)00004-9"},{"key":"e_1_2_1_20_1","volume-title":"The original Borda count and partial voting. Social Choice and Welfare 40, 2 (01","author":"Emerson Peter","year":"2013","unstructured":"Peter Emerson . 2013. The original Borda count and partial voting. Social Choice and Welfare 40, 2 (01 Feb 2013 ), 353--358. Peter Emerson. 2013. The original Borda count and partial voting. Social Choice and Welfare 40, 2 (01 Feb 2013), 353--358."},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1609\/aimag.v30i4.2268"},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1145\/985692.985704"},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1145\/1518701.1518956"},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1080\/10361146.2014.900530"},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1145\/1133265.1133306"},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1145\/1357054.1357252"},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1145\/1449715.1449732"},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1007\/s40593-014-0024-x"},{"key":"e_1_2_1_29_1","volume-title":"Patent US5243697","author":"Hoeber Anthony","year":"1992","unstructured":"Anthony Hoeber , Alan Mandler , and Norman Cox . 1992. Method and apparatus for selecting button functions and retaining selected options on a display , Patent US5243697 , Oracle America Inc. ( 1992 ). https:\/\/patents.google.com\/ patent\/US5243697 Filed March 15th, 1992, Granted July 9th, 1993. Anthony Hoeber, Alan Mandler, and Norman Cox. 1992. Method and apparatus for selecting button functions and retaining selected options on a display, Patent US5243697, Oracle America Inc. (1992). https:\/\/patents.google.com\/ patent\/US5243697 Filed March 15th, 1992, Granted July 9th, 1993."},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1145\/1502650.1502691"},{"volume-title":"Online Controlled Experiments and A\/B Testing","author":"Kohavi Ron","key":"e_1_2_1_31_1","unstructured":"Ron Kohavi and Roger Longbotham . 2017. Online Controlled Experiments and A\/B Testing . Springer US , Boston, MA , 922--929. Ron Kohavi and Roger Longbotham. 2017. Online Controlled Experiments and A\/B Testing. Springer US, Boston, MA, 922--929."},{"key":"e_1_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1145\/223904.223956"},{"key":"e_1_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.ergon.2003.07.004"},{"key":"e_1_2_1_34_1","volume-title":"Proceedings of the 6th ERCIM Workshop \"User Interfaces for All\" (UI4All '00)","author":"Mao Xiaoyang","year":"2000","unstructured":"Xiaoyang Mao , Yuji Hatanaka , Atsumi Imamiya , Yuki Kato , and Kentaro Go . 2000 . Visualizing Computational Wear with Physical Wear . In Proceedings of the 6th ERCIM Workshop \"User Interfaces for All\" (UI4All '00) , Pier-Luigi Emiliani and Constantine Stephanidis (Eds.). CNR-IROE, Pisa, Italy, 12. http:\/\/ui4all.ics.forth.gr\/UI4ALL- 2000\/files\/Long_papers\/ Mao.pdf Xiaoyang Mao, Yuji Hatanaka, Atsumi Imamiya, Yuki Kato, and Kentaro Go. 2000. Visualizing Computational Wear with Physical Wear. In Proceedings of the 6th ERCIM Workshop \"User Interfaces for All\" (UI4All '00), Pier-Luigi Emiliani and Constantine Stephanidis (Eds.). CNR-IROE, Pisa, Italy, 12. http:\/\/ui4all.ics.forth.gr\/UI4ALL-2000\/files\/Long_papers\/ Mao.pdf"},{"key":"e_1_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.1145\/67243.67247"},{"key":"e_1_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.1145\/2212776.2223785"},{"volume-title":"Usability of Adaptable and Adaptive Menus","author":"Park Jungchul","key":"e_1_2_1_37_1","unstructured":"Jungchul Park , Sung H. Han , Yong S. Park , and Youngseok Cho . 2007. Usability of Adaptable and Adaptive Menus . In Usability and Internationalization. HCI and Culture, Nuray Aykin (Ed.). Springer , Berlin, Heidelberg , 405--411. Jungchul Park, Sung H. Han, Yong S. Park, and Youngseok Cho. 2007. Usability of Adaptable and Adaptive Menus. In Usability and Internationalization. HCI and Culture, Nuray Aykin (Ed.). Springer, Berlin, Heidelberg, 405--411."},{"key":"e_1_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jmateco.2015.04.006"},{"key":"e_1_2_1_39_1","volume-title":"Proceedings of the 41st Graphics Interface Conference (GI '15)","author":"Ponsard Antoine","year":"2015","unstructured":"Antoine Ponsard , Kamyar Ardekani , Kailun Zhang , Frederic Ren , Matei Negulescu , and Joanna McGrenere . 2015 . Twist and Pulse: Ephemeral Adaptation to Improve Icon Selection on Smartphones . In Proceedings of the 41st Graphics Interface Conference (GI '15) . Canadian Information Processing Society, Toronto, Ont., Canada, Canada, 219--222. http:\/\/dl.acm.org\/citation.cfm?id=2788890.2788929 Antoine Ponsard, Kamyar Ardekani, Kailun Zhang, Frederic Ren, Matei Negulescu, and Joanna McGrenere. 2015. Twist and Pulse: Ephemeral Adaptation to Improve Icon Selection on Smartphones. In Proceedings of the 41st Graphics Interface Conference (GI '15). Canadian Information Processing Society, Toronto, Ont., Canada, Canada, 219--222. http:\/\/dl.acm.org\/citation.cfm?id=2788890.2788929"},{"key":"e_1_2_1_40_1","doi-asserted-by":"publisher","DOI":"10.1145\/2305484.2305527"},{"key":"e_1_2_1_41_1","doi-asserted-by":"publisher","DOI":"10.1145\/174630.174632"},{"key":"e_1_2_1_42_1","doi-asserted-by":"publisher","DOI":"10.1037\/e577712012-044"},{"key":"e_1_2_1_43_1","first-page":"1356","article-title":"Where's the search? Re-examining user expectations of web objects","volume":"8","author":"Dawn Shaikh A.","year":"2006","unstructured":"A. Dawn Shaikh and K. Lenz . 2006 . Where's the search? Re-examining user expectations of web objects . Usability News 8 , 1 (2006), 1356 -- 1360 . A. Dawn Shaikh and K. Lenz. 2006. Where's the search? Re-examining user expectations of web objects. Usability News 8, 1 (2006), 1356--1360.","journal-title":"Usability News"},{"volume-title":"Proceedings of the XIV Brazilian Symposium on Information Systems (SBSI'18)","author":"Siqueira Jorge Gabriel","key":"e_1_2_1_44_1","unstructured":"Jorge Gabriel Siqueira and Melise M. V . de Paula. 2018. IPEAD A\/B Test Execution Framework . In Proceedings of the XIV Brazilian Symposium on Information Systems (SBSI'18) . ACM, New York, NY, USA, Article 14, 8 pages. Jorge Gabriel Siqueira and Melise M. V. de Paula. 2018. IPEAD A\/B Test Execution Framework. In Proceedings of the XIV Brazilian Symposium on Information Systems (SBSI'18). ACM, New York, NY, USA, Article 14, 8 pages."},{"key":"e_1_2_1_45_1","volume-title":"An Empirical Assessment of Adaptation Techniques. In CHI '05 Extended Abstracts on Human Factors in Computing Systems (CHI EA '05). ACM","author":"Tsandilas Theophanis","year":"2005","unstructured":"Theophanis Tsandilas and m. c. schraefel. 2005 . An Empirical Assessment of Adaptation Techniques. In CHI '05 Extended Abstracts on Human Factors in Computing Systems (CHI EA '05). ACM , New York, NY, USA , 2009--2012. Theophanis Tsandilas and m. c. schraefel. 2005. An Empirical Assessment of Adaptation Techniques. In CHI '05 Extended Abstracts on Human Factors in Computing Systems (CHI EA '05). ACM, New York, NY, USA, 2009--2012."},{"key":"e_1_2_1_46_1","doi-asserted-by":"publisher","DOI":"10.1145\/3172944.3172975"},{"key":"e_1_2_1_47_1","doi-asserted-by":"publisher","DOI":"10.1145\/2390803.2390813"},{"key":"e_1_2_1_48_1","volume-title":"A Methodology for Discovering How to Adaptively Personalize to Users Using Experimental Comparisons. Social Science Research Network","author":"Williams Joseph","year":"2015","unstructured":"Joseph Williams and Neil Heffernan . 2015. A Methodology for Discovering How to Adaptively Personalize to Users Using Experimental Comparisons. Social Science Research Network ( 2015 ). Joseph Williams and Neil Heffernan. 2015. A Methodology for Discovering How to Adaptively Personalize to Users Using Experimental Comparisons. Social Science Research Network (2015)."},{"key":"e_1_2_1_49_1","doi-asserted-by":"publisher","DOI":"10.1145\/3173574.3173781"}],"container-title":["Proceedings of the ACM on Human-Computer Interaction"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3331160","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3331160","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T00:26:07Z","timestamp":1750206367000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3331160"}},"subtitle":["An On-Line A\/B Tester for Comparing User Interface Design Alternatives"],"short-title":[],"issued":{"date-parts":[[2019,6,13]]},"references-count":49,"journal-issue":{"issue":"EICS","published-print":{"date-parts":[[2019,6,13]]}},"alternative-id":["10.1145\/3331160"],"URL":"https:\/\/doi.org\/10.1145\/3331160","relation":{},"ISSN":["2573-0142"],"issn-type":[{"type":"electronic","value":"2573-0142"}],"subject":[],"published":{"date-parts":[[2019,6,13]]},"assertion":[{"value":"2019-06-13","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}