{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,24]],"date-time":"2026-06-24T05:23:44Z","timestamp":1782278624638,"version":"3.54.5"},"reference-count":44,"publisher":"Association for Computing Machinery (ACM)","issue":"2","license":[{"start":{"date-parts":[[2021,4,17]],"date-time":"2021-04-17T00:00:00Z","timestamp":1618617600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/501100000266","name":"Engineering and Physical Sciences Research Council","doi-asserted-by":"crossref","award":["EP\/N020111\/1"],"award-info":[{"award-number":["EP\/N020111\/1"]}],"id":[{"id":"10.13039\/501100000266","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Comput.-Hum. Interact."],"published-print":{"date-parts":[[2021,4,30]]},"abstract":"<jats:p>Cognitive modeling tools have been widely used by researchers and practitioners to help design, evaluate, and study computer user interfaces (UIs). Despite their usefulness, large-scale modeling tasks can still be very challenging due to the amount of manual work needed. To address this scalability challenge, we propose CogTool+, a new cognitive modeling software framework developed on top of the well-known software tool CogTool. CogTool+ addresses the scalability problem by supporting the following key features: (1) a higher level of parameterization and automation; (2) algorithmic components; (3) interfaces for using external data; and (4) a clear separation of tasks, which allows programmers and psychologists to define reusable components (e.g., algorithmic modules and behavioral templates) that can be used by UI\/UX researchers and designers without the need to understand the low-level implementation details of such components. CogTool+ also supports mixed cognitive models required for many large-scale modeling tasks and provides an offline analyzer of simulation results. In order to show how CogTool+ can reduce the human effort required for large-scale modeling, we illustrate how it works using a pedagogical example, and demonstrate its actual performance by applying it to large-scale modeling tasks of two real-world user-authentication systems.<\/jats:p>","DOI":"10.1145\/3447534","type":"journal-article","created":{"date-parts":[[2021,4,17]],"date-time":"2021-04-17T10:12:16Z","timestamp":1618654336000},"page":"1-38","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":1,"title":["CogTool+"],"prefix":"10.1145","volume":"28","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-6084-6719","authenticated-orcid":false,"given":"Haiyue","family":"Yuan","sequence":"first","affiliation":[{"name":"University of Surrey, Guildford, Surrey, United Kingdom"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5628-7328","authenticated-orcid":false,"given":"Shujun","family":"Li","sequence":"additional","affiliation":[{"name":"University of Kent, Canterbury, Kent, United Kingdom"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0095-4061","authenticated-orcid":false,"given":"Patrice","family":"Rusconi","sequence":"additional","affiliation":[{"name":"University of Surrey, Guildford, Surrey, United Kingdom"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2021,4,17]]},"reference":[{"key":"e_1_2_1_1_1","volume-title":"Department of Psychology","author":"ACT-R Research Group","unstructured":"ACT-R Research Group , Department of Psychology , Carnegie Mellon University . [n.d.]. ACT-R. Retrieved from http:\/\/act-r.psy.cmu.edu\/. ACT-R Research Group, Department of Psychology, Carnegie Mellon University. [n.d.]. ACT-R. Retrieved from http:\/\/act-r.psy.cmu.edu\/."},{"key":"e_1_2_1_2_1","doi-asserted-by":"crossref","unstructured":"John R. Anderson. 2007. How Can the Human Mind Occur in the Physical Universe?Oxford University Press.  John R. Anderson. 2007. How Can the Human Mind Occur in the Physical Universe?Oxford University Press.","DOI":"10.1093\/acprof:oso\/9780195324259.001.0001"},{"key":"e_1_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1037\/0033-295X.111.4.1036"},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1145\/1937117.1937118"},{"key":"e_1_2_1_5_1","volume-title":"Proceedings of 2011 33rd International Conference on Software Engineering. IEEE, 691\u2013700","author":"Bellamy Rachel","year":"1985","unstructured":"Rachel Bellamy , Bonnie E. John , and Sandra L. Kogan . 2011. Deploying CogTool: Integrating quantitative usability assessment into real-world software development . In Proceedings of 2011 33rd International Conference on Software Engineering. IEEE, 691\u2013700 . DOI:DOI:https:\/\/doi.org\/10.1145\/ 1985 793.1985890 Rachel Bellamy, Bonnie E. John, and Sandra L. Kogan. 2011. Deploying CogTool: Integrating quantitative usability assessment into real-world software development. In Proceedings of 2011 33rd International Conference on Software Engineering. IEEE, 691\u2013700. DOI:DOI:https:\/\/doi.org\/10.1145\/1985793.1985890"},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1006\/ijhc.2001.0469"},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1145\/358886.358895"},{"key":"e_1_2_1_8_1","volume-title":"Moran","author":"Card Stuart K.","year":"1983","unstructured":"Stuart K. Card , Allen Newell , and Thomas P . Moran . 1983 . The Psychology of Human-Computer Interaction. L. Erlbaum Associates Inc . Stuart K. Card, Allen Newell, and Thomas P. Moran. 1983. The Psychology of Human-Computer Interaction. L. Erlbaum Associates Inc."},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1207\/s15327051hci0803_3"},{"key":"e_1_2_1_10_1","volume-title":"Proceedings of the 2010 SIGCHI Conference on Human Factors in Computing Systems. ACM, 1103\u20131106","author":"Luca Alexander De","year":"2010","unstructured":"Alexander De Luca , Katja Hertzschuch , and Heinrich Hussmann . 2010 . ColorPIN: Securing PIN entry through indirect input . In Proceedings of the 2010 SIGCHI Conference on Human Factors in Computing Systems. ACM, 1103\u20131106 . DOI:DOI:https:\/\/doi.org\/10.1145\/1753326.1753490 Alexander De Luca, Katja Hertzschuch, and Heinrich Hussmann. 2010. ColorPIN: Securing PIN entry through indirect input. In Proceedings of the 2010 SIGCHI Conference on Human Factors in Computing Systems. ACM, 1103\u20131106. DOI:DOI:https:\/\/doi.org\/10.1145\/1753326.1753490"},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-22723-8_9"},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1037\/h0055392"},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1007\/s00779-012-0508-3"},{"key":"e_1_2_1_14_1","unstructured":"Bonnie E. John. [n.d.]. CogTool | Cognitive Crash Dummies: Predictive human performance modeling for UI design. Retrieved from https:\/\/cogtool.wordpress.com\/.  Bonnie E. John. [n.d.]. CogTool | Cognitive Crash Dummies: Predictive human performance modeling for UI design. Retrieved from https:\/\/cogtool.wordpress.com\/."},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1145\/235833.236054"},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1145\/985692.985750"},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-41671-2_21"},{"key":"e_1_2_1_18_1","volume-title":"The Soar Cognitive Architecture","author":"Laird John E.","unstructured":"John E. Laird . 2012. The Soar Cognitive Architecture . MIT Press . John E. Laird. 2012. The Soar Cognitive Architecture. MIT Press."},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.cose.2018.09.003"},{"key":"e_1_2_1_20_1","volume-title":"Proceedings of the Extended Abstracts on Human Factors in Computing Systems. ACM, 1605\u20131608","author":"Luo Lu","unstructured":"Lu Luo and Bonnie E. John . 2005. Predicting task execution time on handheld devices using the keystroke-level model . In Proceedings of the Extended Abstracts on Human Factors in Computing Systems. ACM, 1605\u20131608 . DOI:DOI:https:\/\/doi.org\/10.1145\/1056808.1056977 Lu Luo and Bonnie E. John. 2005. Predicting task execution time on handheld devices using the keystroke-level model. In Proceedings of the Extended Abstracts on Human Factors in Computing Systems. ACM, 1605\u20131608. DOI:DOI:https:\/\/doi.org\/10.1145\/1056808.1056977"},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10515-013-0128-9"},{"key":"e_1_2_1_22_1","unstructured":"Object Refinery Limited. [n.d.]. JFreeChart. Retrieved from http:\/\/www.jfree.org\/jfreechart\/.  Object Refinery Limited. [n.d.]. JFreeChart. Retrieved from http:\/\/www.jfree.org\/jfreechart\/."},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1080\/10447318.2016.1274160"},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1145\/2776891"},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.3758\/BRM.42.3.877"},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1145\/2078827.2078834"},{"key":"e_1_2_1_27_1","doi-asserted-by":"crossref","first-page":"526","DOI":"10.1007\/s00146-006-0050-9","article-title":"Processing: Programming for the media arts","volume":"20","author":"Reas Casey","year":"2006","unstructured":"Casey Reas and Ben Fry . 2006 . Processing: Programming for the media arts . AI & Society: Knowledge, Culture and Communication 20 , 4 (2006), 526 \u2013 538 . DOI:DOI:https:\/\/doi.org\/10.1007\/s00146-006-0050-9 Casey Reas and Ben Fry. 2006. Processing: Programming for the media arts. AI & Society: Knowledge, Culture and Communication 20, 4 (2006), 526\u2013538. DOI:DOI:https:\/\/doi.org\/10.1007\/s00146-006-0050-9","journal-title":"AI & Society: Knowledge, Culture and Communication"},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1145\/1030083.1030116"},{"key":"e_1_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.1145\/365024.365064"},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1145\/1357054.1357085"},{"key":"e_1_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-07308-8_23"},{"key":"e_1_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1145\/2702613.2702960"},{"key":"e_1_2_1_33_1","unstructured":"Soar Research Groups University of Michigan. [n.d.]. Home - Soar Cognitive Architecture. Retrieved from http:\/\/soar.eecs.umich.edu\/.  Soar Research Groups University of Michigan. [n.d.]. Home - Soar Cognitive Architecture. Retrieved from http:\/\/soar.eecs.umich.edu\/."},{"key":"e_1_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.1145\/2207676.2208415"},{"key":"e_1_2_1_35_1","volume-title":"Proceedings of the 2013 International Conference on Software Engineering. IEEE, 152--161","author":"Swearngin Amanda","year":"2013","unstructured":"Amanda Swearngin , Myra B. Cohen , Bonnie E. John , and Rachel K.E. Bellamy . 2013. Human performance regression testing . In Proceedings of the 2013 International Conference on Software Engineering. IEEE, 152--161 . DOI:DOI:https:\/\/doi.org\/10.1109\/ICSE. 2013 .6606561 Amanda Swearngin, Myra B. Cohen, Bonnie E. John, and Rachel K.E. Bellamy. 2013. Human performance regression testing. In Proceedings of the 2013 International Conference on Software Engineering. IEEE, 152--161. DOI:DOI:https:\/\/doi.org\/10.1109\/ICSE.2013.6606561"},{"key":"e_1_2_1_36_1","volume-title":"John","author":"Teo Leonghwee","year":"2008","unstructured":"Leonghwee Teo and Bonnie E . John . 2008 . CogTool-explorer: Towards a tool for predicting user interaction. In Proceedings of the Extended Abstracts on Human Factors in Computing Systems. ACM , 2793--2798. DOI:DOI:https:\/\/doi.org\/10.1145\/1358628.1358763 Leonghwee Teo and Bonnie E. John. 2008. CogTool-explorer: Towards a tool for predicting user interaction. In Proceedings of the Extended Abstracts on Human Factors in Computing Systems. ACM, 2793--2798. DOI:DOI:https:\/\/doi.org\/10.1145\/1358628.1358763"},{"key":"e_1_2_1_37_1","unstructured":"The MITRE Corporation. [n.d.]. Cogulator: A Cognitive Calculator. Retrieved from http:\/\/cogulator.io\/.  The MITRE Corporation. [n.d.]. Cogulator: A Cognitive Calculator. Retrieved from http:\/\/cogulator.io\/."},{"key":"e_1_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.1037\/0096-3445.114.3.285"},{"key":"e_1_2_1_39_1","doi-asserted-by":"publisher","DOI":"10.1145\/2702123.2702212"},{"key":"e_1_2_1_40_1","unstructured":"Wikimedia Foundation Inc.[n.d.]. Lisp (programming language). Retrieved from https:\/\/en.wikipedia.org\/wiki\/Lisp_(programming_language).  Wikimedia Foundation Inc.[n.d.]. Lisp (programming language). Retrieved from https:\/\/en.wikipedia.org\/wiki\/Lisp_(programming_language)."},{"key":"e_1_2_1_41_1","doi-asserted-by":"publisher","DOI":"10.3758\/BF03194406"},{"key":"e_1_2_1_42_1","doi-asserted-by":"publisher","DOI":"10.1080\/13506280500197397"},{"key":"e_1_2_1_43_1","doi-asserted-by":"publisher","DOI":"10.3758\/BF03196569"},{"key":"e_1_2_1_44_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-58460-7_17"}],"container-title":["ACM Transactions on Computer-Human Interaction"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3447534","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3447534","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T20:19:04Z","timestamp":1750191544000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3447534"}},"subtitle":["Modeling Human Performance at Large Scale"],"short-title":[],"issued":{"date-parts":[[2021,4,17]]},"references-count":44,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2021,4,30]]}},"alternative-id":["10.1145\/3447534"],"URL":"https:\/\/doi.org\/10.1145\/3447534","relation":{},"ISSN":["1073-0516","1557-7325"],"issn-type":[{"value":"1073-0516","type":"print"},{"value":"1557-7325","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,4,17]]},"assertion":[{"value":"2018-12-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2021-01-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2021-04-17","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}