{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,21]],"date-time":"2025-10-21T15:09:35Z","timestamp":1761059375749,"version":"3.41.0"},"reference-count":53,"publisher":"Association for Computing Machinery (ACM)","issue":"2","license":[{"start":{"date-parts":[[2012,7,1]],"date-time":"2012-07-01T00:00:00Z","timestamp":1341100800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Comput.-Hum. Interact."],"published-print":{"date-parts":[[2012,7]]},"abstract":"<jats:p>A diversity of user goals and strategies make creation-oriented applications such as word processors or photo-editors difficult to comprehensively test. Evaluating such applications requires testing a large pool of participants to capture the diversity of experience, but traditional usability testing can be prohibitively expensive. To address this problem, this article contributes a new usability evaluation method called backtracking analysis, designed to automate the process of detecting and characterizing usability problems in creation-oriented applications. The key insight is that interaction breakdowns in creation-oriented applications often manifest themselves in backtracking operations that can be automatically logged (e.g., undo and erase operations). Backtracking analysis synchronizes these events to contextual data such as screen capture video, helping the evaluator to characterize specific usability problems. The results from three experiments demonstrate that backtracking events can be effective indicators of usability problems in creation-oriented applications, and can yield a cost-effective alternative to traditional laboratory usability testing.<\/jats:p>","DOI":"10.1145\/2240156.2240164","type":"journal-article","created":{"date-parts":[[2012,8,6]],"date-time":"2012-08-06T18:11:37Z","timestamp":1344276697000},"page":"1-40","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":14,"title":["Backtracking Events as Indicators of Usability Problems in Creation-Oriented Applications"],"prefix":"10.1145","volume":"19","author":[{"given":"David","family":"Akers","sequence":"first","affiliation":[{"name":"University of Puget Sound"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Robin","family":"Jeffries","sequence":"additional","affiliation":[{"name":"Google, Inc."}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Matthew","family":"Simpson","sequence":"additional","affiliation":[{"name":"Google, Inc."}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Terry","family":"Winograd","sequence":"additional","affiliation":[{"name":"Stanford University"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2012,7]]},"reference":[{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1145\/1518701.1518804"},{"key":"e_1_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1006\/ijhc.2000.0441"},{"volume-title":"Psychophysiology: Human Behavior and Physiological Response","year":"2006","author":"Andreassi J. L.","key":"e_1_2_1_4_1"},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1109\/52.84220"},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1145\/1518701.1518948"},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1177\/154193120204602408"},{"key":"e_1_2_1_8_1","unstructured":"Card S. K. Moran T. P. and Newell A. 1983. The Psychology of Human-Computer Interaction. Erlbaum. Card S. K. Moran T. P. and Newell A. 1983. The Psychology of Human-Computer Interaction . Erlbaum."},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1007\/BF02310555"},{"volume-title":"Proceedings of the Human Factors Society. 19--23","author":"Del Galdo E. M.","key":"e_1_2_1_10_1"},{"key":"e_1_2_1_11_1","unstructured":"Dumas J. and Redish J. 1999. A Practical Guide to Usability Testing. Intellect Books. Dumas J. and Redish J. 1999. A Practical Guide to Usability Testing . Intellect Books."},{"key":"e_1_2_1_12_1","unstructured":"Dumas J. S. and Loring B. A. 2008. Moderating Usability Tests: Principles and Practice for Interacting. Morgan Kaufmann. Dumas J. S. and Loring B. A. 2008. Moderating Usability Tests: Principles and Practice for Interacting . Morgan Kaufmann."},{"volume-title":"Classroom in a Book: Adobe Photoshop CS3","author":"Faulkner A.","key":"e_1_2_1_13_1"},{"key":"e_1_2_1_14_1","first-page":"327","article-title":"The critical incident technique","volume":"54","author":"Flanagan J. C.","year":"1954","journal-title":"Psych. Rev."},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1109\/52.595891"},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1145\/1124772.1124961"},{"volume-title":"Proceedings of the 36th Annual Meeting of the Human Factors Society. 1205--1209","author":"Hackman G. S.","key":"e_1_2_1_17_1"},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1145\/948496.948499"},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1145\/238386.238511"},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1207\/S15327590IJHC1304_03"},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1145\/371578.371593"},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.intcom.2008.08.005"},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.5555\/2835557.2835559"},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1145\/503112.503114"},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1145\/286498.286737"},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1080\/014492997119789"},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1145\/985692.985750"},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1145\/142750.142873"},{"key":"e_1_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.1145\/191666.191755"},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1145\/1281192.1281295"},{"key":"e_1_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1145\/985692.985694"},{"key":"e_1_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1177\/001872089403600215"},{"key":"e_1_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.1145\/1240624.1240839"},{"volume-title":"Proceedings of the Usability Professionals Association Conference.","author":"Medlock M. C.","key":"e_1_2_1_34_1"},{"key":"e_1_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.1145\/2331829.2331841"},{"key":"e_1_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.1145\/169059.169166"},{"key":"e_1_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.1145\/97243.97281"},{"key":"e_1_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.1037\/0033-295X.84.3.231"},{"volume-title":"Cognitive engineering","author":"Norman D. A.","key":"e_1_2_1_39_1"},{"key":"e_1_2_1_40_1","first-page":"269","article-title":"Constructive interaction: A method for studying human-computer-human interaction","volume":"84","author":"O\u2019Malley C. E.","year":"1984","journal-title":"Proceedings of IFIP Interact."},{"key":"e_1_2_1_41_1","first-page":"449","article-title":"From novice to expert decision behaviour: A qualitative modelling approach with Petri nets","volume":"20","author":"Rauterberg M.","year":"1995","journal-title":"Adv. Human Factors Ergonomics"},{"key":"e_1_2_1_42_1","doi-asserted-by":"publisher","DOI":"10.1145\/73660.73662"},{"key":"e_1_2_1_43_1","unstructured":"Rubin J. and Chisnell D. 2008. Handbook of Usability Testing. 2nd Ed. Wiley. Rubin J. and Chisnell D. 2008. Handbook of Usability Testing . 2nd Ed. Wiley."},{"key":"e_1_2_1_44_1","doi-asserted-by":"publisher","DOI":"10.1145\/119311.119312"},{"volume-title":"Proceedings of CHI Australia. ACM Press, 1--9.","author":"Skov M. B.","key":"e_1_2_1_45_1"},{"key":"e_1_2_1_46_1","doi-asserted-by":"publisher","DOI":"10.1145\/634067.634236"},{"volume-title":"Proceedings of the Conference of the Centre for Advanced Studies on Collaborative Research.","author":"Swallow J.","key":"e_1_2_1_47_1"},{"key":"e_1_2_1_48_1","first-page":"9","article-title":"Halo 3: How Microsoft Labs invented a new science of play","volume":"15","author":"Thompson C.","year":"2007","journal-title":"Wired"},{"volume-title":"User Experience: Collecting, Analyzing, and Presenting Usability Metrics. Morgan Kaufmann.","year":"2008","author":"Tullis T.","key":"e_1_2_1_49_1"},{"volume-title":"Proceedings of the IEEE International Professional Communication Conference.","author":"Van Den Haak M. J.","key":"e_1_2_1_50_1"},{"key":"e_1_2_1_51_1","doi-asserted-by":"publisher","DOI":"10.1177\/001872089203400407"},{"key":"e_1_2_1_52_1","doi-asserted-by":"publisher","DOI":"10.1145\/142750.142864"},{"key":"e_1_2_1_53_1","unstructured":"Winograd T. and Flores F. 1985. Understanding Computers and Cognition. Ablex. Winograd T. and Flores F. 1985. Understanding Computers and Cognition . Ablex."},{"key":"e_1_2_1_54_1","doi-asserted-by":"publisher","DOI":"10.1145\/838830.838870"}],"container-title":["ACM Transactions on Computer-Human Interaction"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/2240156.2240164","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/2240156.2240164","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T09:20:52Z","timestamp":1750238452000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/2240156.2240164"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2012,7]]},"references-count":53,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2012,7]]}},"alternative-id":["10.1145\/2240156.2240164"],"URL":"https:\/\/doi.org\/10.1145\/2240156.2240164","relation":{},"ISSN":["1073-0516","1557-7325"],"issn-type":[{"type":"print","value":"1073-0516"},{"type":"electronic","value":"1557-7325"}],"subject":[],"published":{"date-parts":[[2012,7]]},"assertion":[{"value":"2011-07-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2012-03-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2012-07-01","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}