{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,7,4]],"date-time":"2025-07-04T05:55:08Z","timestamp":1751608508622},"reference-count":58,"publisher":"Oxford University Press (OUP)","issue":"2","license":[{"start":{"date-parts":[[2021,3,1]],"date-time":"2021-03-01T00:00:00Z","timestamp":1614556800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/academic.oup.com\/journals\/pages\/open_access\/funder_policies\/chorus\/standard_publication_model"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2021,3,12]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:p>To gain a common understanding of an application\u2019s layouts, dialogs and interaction flows, development teams often sketch user interface (UI). Nowadays, they must also define multi-touch gestures, but tools for sketching UIs often lack support for custom gestures and typically just integrate a basic predefined gesture set, which might not suffice to specifically tailor the interaction to the desired use cases. Furthermore, sketching can be enhanced with digital means, but it remains unclear whether digital sketching is actually beneficial when designing gesture-based applications. We extended the AugIR, a digital sketching environment, with GestureCards, a hybrid gesture notation, to allow software engineers to define custom gestures when sketching UIs. We evaluated our approach in a user study contrasting digital and analog sketching of gesture-based UIs.<\/jats:p>","DOI":"10.1093\/iwcomp\/iwab017","type":"journal-article","created":{"date-parts":[[2021,5,12]],"date-time":"2021-05-12T15:49:35Z","timestamp":1620834575000},"page":"134-154","source":"Crossref","is-referenced-by-count":3,"title":["AugIR Meets GestureCards: A Digital Sketching Environment for Gesture-Based Applications"],"prefix":"10.1093","volume":"33","author":[{"given":"Marc","family":"Hesenius","sequence":"first","affiliation":[{"name":"University of Duisburg-Essen, Institute for Software Engineering, Sch\u00fctzenbahn 70, 45127 Essen, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Markus","family":"Kleffmann","sequence":"additional","affiliation":[{"name":"Interaction Room GmbH, Veronikastra\u00dfe 34, 45131 Essen, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Volker","family":"Gruhn","sequence":"additional","affiliation":[{"name":"University of Duisburg-Essen, Institute for Software Engineering, Sch\u00fctzenbahn 70, 45127 Essen, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"286","published-online":{"date-parts":[[2021,5,10]]},"reference":[{"key":"2021091412011424300_ref1","first-page":"1173","article-title":"Documenting natural interactions","volume-title":"CHI \u201813 Extended abstracts on human factors in computing systems, CHI EA \u201813","author":"Altakrouri","year":"2013"},{"key":"2021091412011424300_ref2","first-page":"245","article-title":"A lightweight multistroke recognizer for user interface prototypes","volume-title":"Proc. graphics interface 2010, GI \u201810","author":"Anthony","year":"2010"},{"key":"2021091412011424300_ref3","first-page":"313","article-title":"Are informal tools better? Comparing demais, pencil and paper, and authorware for early multimedia design","volume-title":"Proc. SIGCHI conf. human factors in computing Systems, CHI \u201803","author":"Bailey","year":"2003"},{"key":"2021091412011424300_ref4","doi-asserted-by":"crossref","first-page":"530","DOI":"10.1145\/2635868.2635891","article-title":"Sketches and diagrams in practice","volume-title":"Proc. 22nd ACM SIGSOFT int. symposium on foundations of software engineering, FSE \u201814","author":"Baltes","year":"2014"},{"key":"2021091412011424300_ref5","doi-asserted-by":"crossref","first-page":"30:1","DOI":"10.1145\/2393596.2393630","article-title":"Seeing the forest and the trees: focusing team interaction on value and effort drivers","volume-title":"Proc. ACM SIGSOFT 20th int. symposium on the foundations of software engineering, FSE \u201812","author":"Book","year":"2012"},{"key":"2021091412011424300_ref6","volume-title":"Tamed Agility\u2014Pragmatic Contracting and Collaboration in Agile Software Projects","author":"Book","year":"2016"},{"key":"2021091412011424300_ref7","volume-title":"Sketching User Experiences: Getting the Design Right and the Right Design (Interactive Technologies)","author":"Buxton","year":"2007"},{"key":"2021091412011424300_ref8","first-page":"9","article-title":"Javasketchit: issues in sketching the look of user interfaces","volume-title":"AAAI spring symposium on sketch understanding","author":"Caetano","year":"2002"},{"key":"2021091412011424300_ref9","first-page":"557","article-title":"Let\u2019s go to the whiteboard: how and why software developers use drawings","volume-title":"Proc. SIGCHI conf. human factors in computing systems, CHI \u201807","author":"Cherubini","year":"2007"},{"key":"2021091412011424300_ref10","first-page":"29","article-title":"Inkkit: a generic design tool for the tablet pc","volume-title":"Proc. 6th ACM SIGCHI New Zealand chapter\u2019s int. conf. computer\u2013human interaction: making CHI natural, CHINZ \u201805","author":"Chung","year":"2005"},{"key":"2021091412011424300_ref11","article-title":"Statistical Power Analysis for the Behavioral Sciences","author":"Cohen","year":"1988"},{"key":"2021091412011424300_ref12","doi-asserted-by":"crossref","DOI":"10.1007\/3-540-69706-3_7","article-title":"A room of your own: what do we learn about support of teamwork from assessing teams in dedicated project rooms","volume-title":"Proc. cooperative buildings: integrating information, organization and architecture: first int. workshop, Co\u2019Build \u201898","author":"Covi","year":"1998"},{"key":"2021091412011424300_ref13","doi-asserted-by":"crossref","first-page":"160","DOI":"10.1007\/978-3-540-71876-5_11","article-title":"Sketchixml_ a design tool for informal user interface rapid prototyping","volume-title":"Rapid Integration of Software Engineering Techniques","author":"Coyette","year":"2007"},{"key":"2021091412011424300_ref14","doi-asserted-by":"crossref","DOI":"10.1145\/2494603.2480313","article-title":"Interactive prototyping of tabletop and surface applications","volume-title":"Proc. 5th ACM SIGCHI Symposium on engineering interactive computing systems, EICS \u201813","author":"de Souza Alcantara","year":"2013"},{"key":"2021091412011424300_ref15","first-page":"261","article-title":"Notation and representation in collaborative object-oriented design: an observational study","volume-title":"Proc. 22nd annual ACM SIGPLAN conf. object-oriented programming systems and applications, OOPSLA \u201807","author":"Dekel","year":"2007"},{"key":"2021091412011424300_ref16","doi-asserted-by":"crossref","first-page":"233","DOI":"10.1145\/2148131.2148181","article-title":"GISpL: gestures made easy","volume-title":"Proc. sixth int. conf. Tangible, embedded and embodied interaction, TEI \u201812","author":"Echtler","year":"2012"},{"key":"2021091412011424300_ref17","first-page":"203","article-title":"Monox: extensible gesture notation for mobile devices","volume-title":"Proc. 16th int. conf. human\u2013computer interaction with mobile devices & services, MobileHCI \u201814","author":"Ganh\u00f6r","year":"2014"},{"key":"2021091412011424300_ref18","first-page":"282","article-title":"Supporting generic sketching-based input of diagrams in a domain-specific visual language meta-tool","volume-title":"Proc. 29th int. conf. software engineering, ICSE \u201807","author":"Grundy","year":"2007"},{"key":"2021091412011424300_ref19","first-page":"213","article-title":"Automating UI tests for mobile applications with formal gesture descriptions","volume-title":"Proc. 16th Int. conf. human\u2013computer interaction with mobile devices & services, MobileHCI \u201814","author":"Hesenius","year":"2014"},{"key":"2021091412011424300_ref20","doi-asserted-by":"crossref","first-page":"117","DOI":"10.1145\/2607023.2610278","article-title":"Towards a behavior-oriented specification and testing language for multimodal applications","volume-title":"Proc. 2014 ACM SIGCHI symposium on engineering interactive computing systems, EICS \u201814","author":"Hesenius","year":"2014"},{"key":"2021091412011424300_ref21","doi-asserted-by":"crossref","first-page":"111:1","DOI":"10.1145\/2971485.2996746","article-title":"Introducing gesturecards: a prototyping gesture notation","volume-title":"Proc. 9th Nordic conf. human\u2013computer interaction, NordiCHI \u201816","author":"Hesenius","year":"2016"},{"key":"2021091412011424300_ref22","first-page":"22:1","article-title":"Gesturecards\u2014a hybrid gesture notation","volume-title":"Proc. ACM on human\u2013computer interaction, 3(EICS)","author":"Hesenius","year":"2019"},{"key":"2021091412011424300_ref23","first-page":"327","article-title":"Sketching gesture-based applications in a collaborative working environment with wall-sized displays","volume-title":"2016 int. conf. collaboration technologies and systems, CTS \u201816","author":"Hesenius","year":"2016"},{"key":"2021091412011424300_ref24","doi-asserted-by":"crossref","first-page":"75","DOI":"10.1145\/3102113.3102132","article-title":"A multi-touch-recognizer for gesturecards","volume-title":"Proc. ACM SIGCHI symposium on engineering interactive computing systems, EICS \u201817","author":"Hesenius","year":"2017"},{"key":"2021091412011424300_ref25","first-page":"22","article-title":"Criteria, challenges and opportunities for gesture programming languages","volume-title":"Proc. 1st workshop on engineering gestures for multimodal interfaces","author":"Hoste","year":"2014"},{"key":"2021091412011424300_ref26","volume-title":"Labanotation: The System of Analyzing and Recording Movement","author":"Hutchinson Guest","year":"2005"},{"key":"2021091412011424300_ref27","doi-asserted-by":"crossref","first-page":"201","DOI":"10.1007\/978-1-84800-044-5_8","article-title":"Reporting experiments in software engineering","volume-title":"Guide to Advanced Empirical Software Engineering","author":"Jedlitschka","year":"2008"},{"key":"2021091412011424300_ref28","doi-asserted-by":"crossref","first-page":"527","DOI":"10.1002\/spe.2247","article-title":"Gesture formalization for multitouch","volume":"45","author":"Kammer","year":"2015","journal-title":"Softw. Pract. Exp."},{"key":"2021091412011424300_ref29","doi-asserted-by":"crossref","first-page":"2:1","DOI":"10.1145\/2060329.2060339","article-title":"A domain specific language to define gestures for multi-touch applications","volume-title":"Proc. 10th workshop on domain-specific modeling, DSM \u201810","author":"Khandkar","year":"2010"},{"key":"2021091412011424300_ref30","doi-asserted-by":"crossref","first-page":"269","DOI":"10.1145\/1936652.1936710","article-title":"A language to define multi-touch interactions","volume-title":"ACM Int. conf. interactive tabletops and surfaces, ITS \u201810","author":"Khandkar","year":"2010"},{"key":"2021091412011424300_ref31","first-page":"267","article-title":"EventHurdle: supporting designers\u2019 exploratory interaction prototyping with gesture-based sensors","volume-title":"Proc. SIGCHI conf. human factors in computing systems, CHI \u201813","author":"Kim","year":"2013"},{"key":"2021091412011424300_ref32","doi-asserted-by":"crossref","first-page":"477","DOI":"10.1145\/2380116.2380176","article-title":"Proton++: a customizable declarative multitouch framework","volume-title":"Proc. 25th annual ACM symposium on user interface software and technology","author":"Kin","year":"2012"},{"key":"2021091412011424300_ref33","doi-asserted-by":"crossref","first-page":"2885","DOI":"10.1145\/2207676.2208694","article-title":"Proton: multitouch gestures as regular expressions","volume-title":"Proc. SIGCHI conf. human factors in computing systems","author":"Kin","year":"2012"},{"key":"2021091412011424300_ref34","doi-asserted-by":"crossref","first-page":"721","DOI":"10.1109\/TSE.2002.1027796","article-title":"Preliminary guidelines for empirical research in software engineering","volume":"28","author":"Kitchenham","year":"2002","journal-title":"IEEE Trans. Softw. Eng."},{"key":"2021091412011424300_ref35","first-page":"883","article-title":"AugIR: the conceptual design and evaluation of an augmented interaction room","volume-title":"Proc. 29th ACM\/IEEE int. conf. automated software engineering, ASE \u201814","author":"Kleffmann","year":"2014"},{"key":"2021091412011424300_ref36","first-page":"9","article-title":"Supporting collaboration of heterogeneous teams in an augmented team room","volume-title":"Proc. 6th int. workshop on social software engineering, SSE 2014","author":"Kleffmann","year":"2014"},{"key":"2021091412011424300_ref37","doi-asserted-by":"crossref","first-page":"200","DOI":"10.1145\/2774225.2775076","article-title":"Connecting UI and business processes in a collaborative sketching environment","volume-title":"Proc. 7th ACM SIGCHI symposium on engineering interactive computing systems, EICS \u201815","author":"Kleffmann","year":"2015"},{"key":"2021091412011424300_ref38","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1007\/s10515-017-0221-6","article-title":"Evaluation of a traceability approach for informal freehand sketches","volume":"25","author":"Kleffmann","year":"2018","journal-title":"Automat. Softw. Eng."},{"key":"2021091412011424300_ref39","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1109\/SST.2015.8","article-title":"Establishing and navigating trace links between elements of informal diagram sketches","volume-title":"2015 IEEE\/ACM 8th int. symposium on software and systems traceability","author":"Kleffmann","year":"2015"},{"key":"2021091412011424300_ref40","first-page":"398","article-title":"Silk: sketching interfaces like krazy","volume-title":"Conf. companion on human factors in computing systems, CHI \u201896","author":"Landay","year":"1996"},{"key":"2021091412011424300_ref41","doi-asserted-by":"crossref","first-page":"56","DOI":"10.1109\/2.910894","article-title":"Sketching interfaces: toward more human interface design","volume":"34","author":"Landay","year":"2001","journal-title":"Computer"},{"key":"2021091412011424300_ref42","doi-asserted-by":"crossref","first-page":"65","DOI":"10.1111\/j.1551-6708.1987.tb00863.x","article-title":"Why a diagram is (sometimes) worth ten thousand words","volume":"11","author":"Larkin","year":"1987","journal-title":"Cogn. Sci."},{"key":"2021091412011424300_ref43","first-page":"492","article-title":"Maintaining mental models: a study of developer work habits","volume-title":"Proc. 28th int. conf. software engineering, ICSE \u201806","author":"LaToza","year":"2006"},{"key":"2021091412011424300_ref44","first-page":"307","article-title":"A visual language for sketching large and complex interactive designs","volume-title":"Proc. SIGCHI conf. human factors in computing systems, CHI \u201802","author":"Lin","year":"2002"},{"key":"2021091412011424300_ref45","first-page":"1685","article-title":"Gesture Script: recognizing gestures and their structure using rendering scripts and interactively trained parts","volume-title":"Proc. 32nd annual ACM conf. human factors in computing systems\u2014CHI \u201814","author":"L\u00fc","year":"2014"},{"key":"2021091412011424300_ref46","first-page":"257","article-title":"Gesture studio: authoring multi-touch interactions through demonstration and declaration","volume-title":"Proc. SIGCHI conf. human factors in computing systems, CHI \u201813","author":"L\u00fc","year":"2013"},{"key":"2021091412011424300_ref47","first-page":"331","article-title":"Supporting informal design with interactive whiteboards","volume-title":"Proc. SIGCHI conf. human factors in computing systems, CHI \u201814","author":"Mangano","year":"2014"},{"key":"2021091412011424300_ref48","first-page":"161","article-title":"Inspector\u2014interactive UI specification tool","volume-title":"Proc. 7th int. conf. computer aided design of user interfaces (CADUI) 2008","author":"Memmel","year":"2008"},{"key":"2021091412011424300_ref49","doi-asserted-by":"crossref","first-page":"259","DOI":"10.1207\/S15327051HCI1803_3","article-title":"Denim: an informal web site design tool inspired by observations of practice","volume":"18","author":"Newman","year":"2003","journal-title":"Hum.-Comput. Interact."},{"key":"2021091412011424300_ref50","doi-asserted-by":"crossref","first-page":"4:1","DOI":"10.1145\/1959022.1959026","article-title":"Sketching interactive systems with sketchify","volume":"18","author":"Obrenovic","year":"2011","journal-title":"ACM Trans. Comput.-Hum. Interact"},{"key":"2021091412011424300_ref51","first-page":"1337","article-title":"Interacting with sketched interface designs: an evaluation study","volume-title":"CHI \u201804 extended abstracts on human factors in computing systems, CHI EA \u201804","author":"Plimmer","year":"2004"},{"key":"2021091412011424300_ref52","first-page":"31","article-title":"Beautifying sketching-based design tool content: issues and experiences","volume-title":"Proc. sixth Australasian conf. user interface\u2014volume 40, AUIC \u201805","author":"Plimmer","year":"2005"},{"key":"2021091412011424300_ref53","doi-asserted-by":"crossref","first-page":"88","DOI":"10.1007\/s00163-002-0028-7","article-title":"Support value of sketching in the design process","volume":"14","author":"Sch\u00fctze","year":"2003","journal-title":"Res. Eng. Design"},{"key":"2021091412011424300_ref54","doi-asserted-by":"crossref","first-page":"333","DOI":"10.1023\/A:1008716330212","article-title":"Formality considered harmful: experiences, emerging themes, and directions on the use of formal representations in interactive systems","volume":"8","author":"Shipman","year":"1999","journal-title":"Comput. Support. Coop. Work"},{"key":"2021091412011424300_ref55","first-page":"506","article-title":"Roomware: the second generation","volume-title":"CHI \u201802 extended abstracts on human factors in computing systems, CHI EA \u201802","author":"Streitz","year":"2002"},{"key":"2021091412011424300_ref56","first-page":"339","article-title":"How does radical collocation help a team succeed?","volume-title":"Proc. 2000 ACM conf. computer supported cooperative work, CSCW \u201800","author":"Teasley","year":"2000"},{"key":"2021091412011424300_ref57","doi-asserted-by":"crossref","first-page":"273","DOI":"10.1145\/2388676.2388732","article-title":"Gestures as point clouds","volume-title":"Proc. 14th ACM int. conf. multimodal interaction\u2014ICMI \u201812","author":"Vatavu","year":"2012"},{"key":"2021091412011424300_ref58","doi-asserted-by":"crossref","first-page":"159","DOI":"10.1145\/1294211.1294238","article-title":"Gestures without libraries, toolkits or training: a $1 recognizer for user interface prototypes","volume-title":"Proc. 20th annual ACM symp. user interface software and technology, UIST \u201807","author":"Wobbrock","year":"2007"}],"container-title":["Interacting with Computers"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/academic.oup.com\/iwc\/article-pdf\/33\/2\/134\/40358790\/iwab017.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"http:\/\/academic.oup.com\/iwc\/article-pdf\/33\/2\/134\/40358790\/iwab017.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,9,14]],"date-time":"2021-09-14T12:04:21Z","timestamp":1631621061000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/iwc\/article\/33\/2\/134\/6273339"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,3]]},"references-count":58,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2021,5,10]]},"published-print":{"date-parts":[[2021,3,12]]}},"URL":"https:\/\/doi.org\/10.1093\/iwcomp\/iwab017","relation":{},"ISSN":["0953-5438","1873-7951"],"issn-type":[{"value":"0953-5438","type":"print"},{"value":"1873-7951","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2021,3]]},"published":{"date-parts":[[2021,3]]}}}