{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,3,11]],"date-time":"2025-03-11T04:36:36Z","timestamp":1741667796200,"version":"3.38.0"},"reference-count":49,"publisher":"SAGE Publications","issue":"3","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["AIC"],"published-print":{"date-parts":[[2022,9,5]]},"abstract":"<jats:p>Humans interact more and more with systems containing AI components. In this work, we focus on hand gestures such as handwriting and sketches serving as inputs to such systems. They are represented as a trajectory, i.e. sequence of points, that is altered to improve interaction with an AI model while keeping the model fixed. Optimized inputs are accompanied by instructions on how to create them. We aim to cut on effort for humans and recognition errors while limiting changes to original inputs. We derive multiple objectives and measures and propose continuous and discrete optimization methods embracing the AI model to improve samples in an iterative fashion by removing, shifting and reordering points of the gesture trajectory. Our quantitative and qualitative evaluation shows that mimicking generated proposals that differ only modestly from the original ones leads to lower error rates and requires less effort. Furthermore, our work can be easily adjusted for sketch abstraction improving on prior work.<\/jats:p>","DOI":"10.3233\/aic-210081","type":"journal-article","created":{"date-parts":[[2022,5,6]],"date-time":"2022-05-06T15:17:48Z","timestamp":1651850268000},"page":"153-169","source":"Crossref","is-referenced-by-count":2,"title":["Optimizing human hand gestures for AI-systems"],"prefix":"10.1177","volume":"35","author":[{"given":"Johannes","family":"Schneider","sequence":"first","affiliation":[{"name":"Information Systems, University of Liechtenstein, Liechtenstein"}]}],"member":"179","reference":[{"key":"10.3233\/AIC-210081_ref1","doi-asserted-by":"crossref","unstructured":"S.\u00a0Amershi, D.\u00a0Weld, M.\u00a0Vorvoreanu, A.\u00a0Fourney, B.\u00a0Nushi, P.\u00a0Collisson, J.\u00a0Suh, S.\u00a0Iqbal, P.N.\u00a0Bennett, K.\u00a0Inkpen et al., Guidelines for human-AI interaction, in: Proc. of the CHI Conference on Human Factors in Computing Systems, 2019.","DOI":"10.1145\/3290605.3300233"},{"key":"10.3233\/AIC-210081_ref2","doi-asserted-by":"crossref","unstructured":"G.\u00a0Bansal, B.\u00a0Nushi, E.\u00a0Kamar, W.S.\u00a0Lasecki, D.S.\u00a0Weld and E.\u00a0Horvitz, Beyond accuracy: The role of mental models in human-AI team performance, in: Proceedings of the AAAI Conference on Human Computation and Crowdsourcing, 2019.","DOI":"10.1609\/hcomp.v7i1.5285"},{"key":"10.3233\/AIC-210081_ref3","doi-asserted-by":"crossref","unstructured":"G.\u00a0Bansal, B.\u00a0Nushi, E.\u00a0Kamar, D.S.\u00a0Weld, W.S.\u00a0Lasecki and E.\u00a0Horvitz, Updates in human-AI teams: Understanding and addressing the performance\/compatibility tradeoff, in: Proc. of the AAAI Conference on Artificial Intelligence, 2019.","DOI":"10.1609\/aaai.v33i01.33012429"},{"key":"10.3233\/AIC-210081_ref4","doi-asserted-by":"crossref","unstructured":"J.\u00a0Bao, D.\u00a0Chen, F.\u00a0Wen, H.\u00a0Li and G.\u00a0Hua, CVAE-GAN: Fine-grained image generation through asymmetric training, in: Proc. of the Int. Conf. on Computer Vision, 2017.","DOI":"10.1109\/ICCV.2017.299"},{"key":"10.3233\/AIC-210081_ref5","doi-asserted-by":"crossref","unstructured":"C.\u00a0Bartneck and J.\u00a0Forlizzi, A design-centred framework for social human-robot interaction, in: Workshop on Robot and Human Interactive Communication, 2004.","DOI":"10.1145\/985921.986205"},{"key":"10.3233\/AIC-210081_ref6","doi-asserted-by":"crossref","unstructured":"M.\u00a0Basalla, J.\u00a0Schneider and J.\u00a0vom Brocke, Creativity of deep learning: Conceptualization and assessment, in: Proceedings of the 14th International Conference on Agents and Artificial Intelligence, 2022.","DOI":"10.5220\/0010783500003116"},{"key":"10.3233\/AIC-210081_ref7","unstructured":"O.\u00a0Bastani, Y.\u00a0Ioannou, L.\u00a0Lampropoulos, D.\u00a0Vytiniotis, A.\u00a0Nori and A.\u00a0Criminisi, Measuring neural net robustness with constraints, in: Advances in Neural Information Processing Systems, 2016."},{"key":"10.3233\/AIC-210081_ref8","unstructured":"A.\u00a0Billard and K.\u00a0Dautenhahn, Grounding communication in situated, social robots, in: Proceedings Towards Intelligent Mobile Robots Conference, Report No. UMCS-97-9-1, Department of Computer Science, Manchester University, 1997."},{"key":"10.3233\/AIC-210081_ref9","doi-asserted-by":"crossref","unstructured":"Y.\u00a0Bisk, D.\u00a0Yuret and D.\u00a0Marcu, Natural language communication with robots, in: Proc. of Conf. of the North American Chapter of the Ass. for Computational Linguistics: Human Language Technologies, 2016.","DOI":"10.18653\/v1\/N16-1089"},{"key":"10.3233\/AIC-210081_ref10","doi-asserted-by":"crossref","unstructured":"C.\u00a0Breazeal, C.D.\u00a0Kidd, A.L.\u00a0Thomaz, G.\u00a0Hoffman and M.\u00a0Berlin, Effects of nonverbal communication on efficiency and robustness in human-robot teamwork, in: Int. Conf. on Intelligent Robots and Systems, 2005.","DOI":"10.1109\/IROS.2005.1545011"},{"key":"10.3233\/AIC-210081_ref11","doi-asserted-by":"crossref","unstructured":"J.\u00a0Calvo-Zaragoza and J.\u00a0Oncina, Recognition of pen-based music notation: The HOMUS dataset, in: Int. Conf. on Pattern Recognition, 2014.","DOI":"10.1109\/ICPR.2014.524"},{"key":"10.3233\/AIC-210081_ref12","unstructured":"M.\u00a0Carroll, R.\u00a0Shah, M.K.\u00a0Ho, T.\u00a0Griffiths, S.\u00a0Seshia, P.\u00a0Abbeel and A.\u00a0Dragan, On the utility of learning about humans for human-AI coordination, in: Adv. in Neural Information Processing Systems, 2019."},{"key":"10.3233\/AIC-210081_ref14","doi-asserted-by":"publisher","first-page":"539","DOI":"10.1016\/j.future.2018.01.055","article-title":"In the shades of the uncanny valley: An experimental study of human-chatbot interaction","volume":"92","author":"Ciechanowski","year":"2019","journal-title":"Future Generation Computer Systems"},{"key":"10.3233\/AIC-210081_ref15","unstructured":"A.\u00a0Dhurandhar, P.-Y.\u00a0Chen, R.\u00a0Luss, C.-C.\u00a0Tu, P.\u00a0Ting, K.\u00a0Shanmugam and P.\u00a0Das, Explanations based on the missing: Towards contrastive explanations with pertinent negatives, in: Advances in Neural Information Processing Systems, 2018."},{"key":"10.3233\/AIC-210081_ref16","doi-asserted-by":"publisher","DOI":"10.1088\/1741-2552\/aae069"},{"issue":"4","key":"10.3233\/AIC-210081_ref17","first-page":"1","article-title":"How do humans sketch objects?","volume":"31","author":"Eitz","year":"2012","journal-title":"ACM Transactions on graphics (TOG)"},{"issue":"3","key":"10.3233\/AIC-210081_ref18","doi-asserted-by":"publisher","first-page":"199","DOI":"10.1111\/j.1440-1630.2005.00505.x","article-title":"Improving handwriting without teaching handwriting: The consultative clinical reasoning process","volume":"52","author":"Erhardt","year":"2005","journal-title":"Australian Occupational Therapy Journal"},{"key":"10.3233\/AIC-210081_ref19","doi-asserted-by":"crossref","unstructured":"F.\u00a0Fusco, M.\u00a0Vlachos, V.\u00a0Vasileiadis, K.\u00a0Wardatzky and J.\u00a0Schneider, Reconet: An interpretable neural architecture for recommender systems, in: Proc of Int. Joint Conf. on Artificial Intelligence (IJCAI), 2019.","DOI":"10.24963\/ijcai.2019\/325"},{"key":"10.3233\/AIC-210081_ref20","doi-asserted-by":"crossref","unstructured":"P.\u00a0Gallina, N.\u00a0Bellotto and M.\u00a0Di Luca, Progressive co-adaptation in human-machine interaction, in: 2015 12th International Conference on Informatics in Control, Automation and Robotics (ICINCO), Vol.\u00a02, 2015, pp.\u00a0362\u2013368.","DOI":"10.5220\/0005561003620368"},{"key":"10.3233\/AIC-210081_ref27","doi-asserted-by":"crossref","unstructured":"J.\u00a0Hois, D.\u00a0Theofanou-Fuelbier and A.J.\u00a0Junk, How to achieve explainability and transparency in human AI interaction, in: Int. Conference on Human-Computer Interaction, 2019.","DOI":"10.1007\/978-3-030-23528-4_25"},{"issue":"1","key":"10.3233\/AIC-210081_ref28","doi-asserted-by":"publisher","first-page":"13","DOI":"10.2182\/cjot.2011.78.1.3","article-title":"A systematic review of interventions to improve handwriting","volume":"78","author":"Hoy","year":"2011","journal-title":"Canadian Journal of Occupational Therapy"},{"key":"10.3233\/AIC-210081_ref29","doi-asserted-by":"publisher","first-page":"99","DOI":"10.1016\/j.ijhcs.2019.05.006","article-title":"History and future of human-automation interaction","volume":"131","author":"Janssen","year":"2019","journal-title":"International journal of human-computer studies"},{"key":"10.3233\/AIC-210081_ref30","doi-asserted-by":"publisher","DOI":"10.1109\/ICFHR.2014.76"},{"key":"10.3233\/AIC-210081_ref31","doi-asserted-by":"publisher","DOI":"10.1109\/SOCPAR.2014.7008025"},{"issue":"1","key":"10.3233\/AIC-210081_ref32","doi-asserted-by":"publisher","first-page":"183","DOI":"10.1146\/annurev.anthro.32.061002.093440","article-title":"Complex adaptive systems","volume":"32","author":"Lansing","year":"2003","journal-title":"Annual review of anthropology"},{"key":"10.3233\/AIC-210081_ref33","doi-asserted-by":"crossref","unstructured":"F.\u00a0Liu, X.\u00a0Deng, Y.-K.\u00a0Lai, Y.-J.\u00a0Liu, C.\u00a0Ma and H.\u00a0Wang, Sketchgan: Joint sketch completion and recognition with generative adversarial network, in: Proc. of the Conference on Computer Vision and Pattern Recognition, 2019.","DOI":"10.1109\/CVPR.2019.00598"},{"issue":"4","key":"10.3233\/AIC-210081_ref34","doi-asserted-by":"publisher","first-page":"535","DOI":"10.1007\/s12599-019-00600-8","article-title":"AI-based digital assistants","volume":"61","author":"Maedche","year":"2019","journal-title":"Business & Information Systems Engineering"},{"key":"10.3233\/AIC-210081_ref35","doi-asserted-by":"crossref","unstructured":"N.\u00a0Malkin, M.\u00a0Harbach, A.\u00a0De Luca and S.\u00a0Egelman, The anatomy of smartphone unlocking: Why and how Android users around the world lock their phones, in: GetMobile: Mobile Computing and Communications, 2017.","DOI":"10.1145\/2858036.2858267"},{"issue":"1","key":"10.3233\/AIC-210081_ref36","doi-asserted-by":"publisher","first-page":"185","DOI":"10.1007\/s12369-018-0485-4","article-title":"User-adaptive interaction in social robots: A survey focusing on non-physical interaction","volume":"11","author":"Martins","year":"2019","journal-title":"International Journal of Social Robotics"},{"key":"10.3233\/AIC-210081_ref37","doi-asserted-by":"crossref","first-page":"53","DOI":"10.1080\/10580530.2020.1849465","article-title":"Explainable artificial intelligence: Objectives, stakeholders, and future research opportunities","volume":"39","author":"Meske","year":"2022","journal-title":"Information Systems Management"},{"key":"10.3233\/AIC-210081_ref38","doi-asserted-by":"crossref","unstructured":"U.R.\u00a0Muhammad, Y.\u00a0Yang, T.M.\u00a0Hospedales, T.\u00a0Xiang and Y.-Z.\u00a0Song, Goal-driven sequential data abstraction, in: Proc. of the International Conference on Computer Vision, 2019.","DOI":"10.1109\/ICCV.2019.00016"},{"key":"10.3233\/AIC-210081_ref39","unstructured":"A.I.\u00a0Niculescu and R.E.\u00a0Banchs, Strategies to cope with errors in human-machine spoken interactions: Using chatbots as back-off mechanism for task-oriented dialogues, in: Proc. Errors by Humans and Machines in Multimedia, Multimodal and Multilingual Data Processing (ERRARE), 2015."},{"key":"10.3233\/AIC-210081_ref40","doi-asserted-by":"publisher","DOI":"10.3390\/robotics8030054"},{"key":"10.3233\/AIC-210081_ref41","doi-asserted-by":"publisher","DOI":"10.23915\/distill.00007"},{"key":"10.3233\/AIC-210081_ref42","doi-asserted-by":"crossref","unstructured":"O.\u00a0Poursaeed, I.\u00a0Katsman, B.\u00a0Gao and S.\u00a0Belongie, Generative adversarial perturbations, in: Pro. of Conference on Computer Vision and Pattern Recognition, 2018.","DOI":"10.1109\/CVPR.2018.00465"},{"key":"10.3233\/AIC-210081_ref43","doi-asserted-by":"crossref","unstructured":"U.\u00a0Riaz Muhammad, Y.\u00a0Yang, Y.-Z.\u00a0Song, T.\u00a0Xiang and T.M.\u00a0Hospedales, Learning deep sketch abstraction, in: Proc. of the Conference on Computer Vision and Pattern Recognition, 2018.","DOI":"10.1109\/CVPR.2018.00836"},{"key":"10.3233\/AIC-210081_ref44","unstructured":"C.\u00a0Rzepka and B.\u00a0Berger, User interaction with AI-enabled systems: A systematic review of IS research, in: Int. Conf. on Information Systems (ICIS), 2018."},{"key":"10.3233\/AIC-210081_ref45","doi-asserted-by":"crossref","unstructured":"J.\u00a0Schneider, Human-to-AI coach: Improving human inputs to AI systems, in: International Symposium on Intelligent Data Analysis, 2020.","DOI":"10.1007\/978-3-030-44584-3_34"},{"key":"10.3233\/AIC-210081_ref46","unstructured":"J.\u00a0Schneider and J.\u00a0Handali, Personalized explanation in machine learning, in: European Conference on Information Systems (ECIS), 2019."},{"key":"10.3233\/AIC-210081_ref47","doi-asserted-by":"crossref","unstructured":"J.\u00a0Schneider and M.\u00a0Vlachos, Personalization of deep learning, in: Data Science \u2013 Analytics and Applications, 2020.","DOI":"10.1007\/978-3-658-32182-6_14"},{"key":"10.3233\/AIC-210081_ref48","doi-asserted-by":"crossref","unstructured":"J.\u00a0Schneider and M.\u00a0Vlachos, Explaining neural networks by decoding layer activations, in: International Symposium on Intelligent Data Analysis, 2021, pp.\u00a063\u201375.","DOI":"10.1007\/978-3-030-74251-5_6"},{"issue":"2","key":"10.3233\/AIC-210081_ref50","first-page":"2","article-title":"Research perspectives: The rise of human machines: How cognitive computing systems challenge assumptions of user-system interaction","volume":"21","author":"Schuetz","year":"2020","journal-title":"Journal of the Association for Information Systems"},{"key":"10.3233\/AIC-210081_ref51","doi-asserted-by":"publisher","DOI":"10.1145\/2500423.2500434"},{"issue":"6","key":"10.3233\/AIC-210081_ref52","doi-asserted-by":"publisher","first-page":"495","DOI":"10.1080\/10447318.2020.1741118","article-title":"Human-centered artificial intelligence: Reliable, safe & trustworthy","volume":"36","author":"Shneiderman","year":"2020","journal-title":"International Journal of Human-Computer Interaction"},{"key":"10.3233\/AIC-210081_ref54","doi-asserted-by":"crossref","unstructured":"V.\u00a0Venkatesh, M.G.\u00a0Morris, G.B.\u00a0Davis and F.D.\u00a0Davis, User acceptance of information technology: Toward a unified view, in: MIS Quarterly, 2003.","DOI":"10.2307\/30036540"},{"issue":"3","key":"10.3233\/AIC-210081_ref55","doi-asserted-by":"publisher","first-page":"281","DOI":"10.1006\/ijhc.2001.0499","article-title":"Interactive machine learning: Letting users build classifiers","volume":"55","author":"Ware","year":"2001","journal-title":"International Journal of Human-Computer Studies"},{"issue":"3","key":"10.3233\/AIC-210081_ref57","doi-asserted-by":"publisher","first-page":"411","DOI":"10.1007\/s11263-016-0932-3","article-title":"Sketch-a-Net: A deep neural network that beats humans","volume":"122","author":"Yu","year":"2017","journal-title":"International journal of computer vision"},{"issue":"5","key":"10.3233\/AIC-210081_ref58","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3178115","article-title":"Deep learning for environmentally robust speech recognition: An overview of recent developments","volume":"9","author":"Zhang","year":"2018","journal-title":"ACM Transactions on Intelligent Systems and Technology (TIST)"},{"key":"10.3233\/AIC-210081_ref61","doi-asserted-by":"crossref","unstructured":"J.-Y.\u00a0Zhu, P.\u00a0Kr\u00e4henb\u00fchl, E.\u00a0Shechtman and A.\u00a0Efros, Generative visual manipulation on the natural image manifold, in: European Conf. on Computer Vision, 2016.","DOI":"10.1007\/978-3-319-46454-1_36"}],"container-title":["AI Communications"],"original-title":[],"link":[{"URL":"https:\/\/content.iospress.com\/download?id=10.3233\/AIC-210081","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,3,10]],"date-time":"2025-03-10T22:22:20Z","timestamp":1741645340000},"score":1,"resource":{"primary":{"URL":"https:\/\/journals.sagepub.com\/doi\/full\/10.3233\/AIC-210081"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,9,5]]},"references-count":49,"journal-issue":{"issue":"3"},"URL":"https:\/\/doi.org\/10.3233\/aic-210081","relation":{},"ISSN":["1875-8452","0921-7126"],"issn-type":[{"type":"electronic","value":"1875-8452"},{"type":"print","value":"0921-7126"}],"subject":[],"published":{"date-parts":[[2022,9,5]]}}}