{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,10]],"date-time":"2026-04-10T08:16:20Z","timestamp":1775808980107,"version":"3.50.1"},"reference-count":75,"publisher":"Association for Computing Machinery (ACM)","issue":"CSCW","license":[{"start":{"date-parts":[[2018,11,1]],"date-time":"2018-11-01T00:00:00Z","timestamp":1541030400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/501100001807","name":"FAPESP","doi-asserted-by":"crossref","award":["2015\/24527-3"],"award-info":[{"award-number":["2015\/24527-3"]}],"id":[{"id":"10.13039\/501100001807","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/100008883","name":"Northern Arizona University","doi-asserted-by":"crossref","id":[{"id":"10.13039\/100008883","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/501100003593","name":"CNPq","doi-asserted-by":"crossref","award":["430642\/2016-4"],"award-info":[{"award-number":["430642\/2016-4"]}],"id":[{"id":"10.13039\/501100003593","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Proc. ACM Hum.-Comput. Interact."],"published-print":{"date-parts":[[2018,11]]},"abstract":"<jats:p>Leveraging the pull request model of social coding platforms, Open Source Software (OSS) integrators review developers' contributions, checking aspects like license, code quality, and testability. Some projects use bots to automate predefined, sometimes repetitive tasks, thereby assisting integrators' and contributors' work. Our research investigates the usage and impact of such bots. We sampled 351 popular projects from GitHub and found that 93 (26%) use bots. We classified the bots, collected metrics from before and after bot adoption, and surveyed 228 developers and integrators. Our results indicate that bots perform numerous tasks. Although integrators reported that bots are useful for maintenance tasks, we did not find a consistent, statistically significant difference between before and after bot adoption across the analyzed projects in terms of number of comments, commits, changed files, and time to close pull requests. Our survey respondents deem the current bots as not smart enough and provided insights into the bots' relevance for specific tasks, challenges, and potential new features. We discuss some of the raised suggestions and challenges in light of the literature in order to help GitHub bot designers reuse and test ideas and technologies already investigated in other contexts.<\/jats:p>","DOI":"10.1145\/3274451","type":"journal-article","created":{"date-parts":[[2018,11,1]],"date-time":"2018-11-01T21:21:27Z","timestamp":1541107287000},"page":"1-19","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":138,"title":["The Power of Bots"],"prefix":"10.1145","volume":"2","author":[{"given":"Mairieli","family":"Wessel","sequence":"first","affiliation":[{"name":"University of S\u00e3o Paulo, S\u00e3o Paulo, Brazil"}]},{"given":"Bruno Mendes","family":"de Souza","sequence":"additional","affiliation":[{"name":"Federal University of Technology, Paran\u00e1, Campo Mour\u00e3o, Brazil"}]},{"given":"Igor","family":"Steinmacher","sequence":"additional","affiliation":[{"name":"Northern Arizona University &amp; Federal University of Technology, Paran\u00e1, Flagstaff, AZ, USA"}]},{"given":"Igor S.","family":"Wiese","sequence":"additional","affiliation":[{"name":"Federal University of Technology, Paran\u00e1, Campo Mour\u00e3o, PR, Brazil"}]},{"given":"Ivanilton","family":"Polato","sequence":"additional","affiliation":[{"name":"Federal University of Technology, Paran\u00e1, Campo Mour\u00e3o, PR, Brazil"}]},{"given":"Ana Paula","family":"Chaves","sequence":"additional","affiliation":[{"name":"Federal University of Technology, Paran\u00e1 &amp; Northern Arizona University, Campo Mour\u00e3o, PR, Brazil"}]},{"given":"Marco A.","family":"Gerosa","sequence":"additional","affiliation":[{"name":"Northern Arizona University, Flagstaff, AZ, USA"}]}],"member":"320","published-online":{"date-parts":[[2018,11]]},"reference":[{"key":"e_1_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1145\/2675133.2675208"},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1002\/smr.1821"},{"key":"e_1_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1145\/1134285.1134336"},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1145\/2591708.2591728"},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE-NIER.2017.17"},{"key":"e_1_2_1_6_1","first-page":"218","article-title":"Chatbot technology: A possible means of unlocking student potential to learn how to learn","volume":"4","author":"Bii Patrick","year":"2013","unstructured":"Patrick Bii. 2013. Chatbot technology: A possible means of unlocking student potential to learn how to learn. Educational Research 4, 2 (2013), 218--221.","journal-title":"Educational Research"},{"key":"e_1_2_1_7_1","volume-title":"Understanding the Factors That Impact the Popularity of GitHub Repositories. In 2016 IEEE International Conference on Software Maintenance and Evolution (ICSME). IEEE","author":"Borges Hudson","year":"2016","unstructured":"Hudson Borges, Andre Hora, and Marco Tulio Valente. 2016. Understanding the Factors That Impact the Popularity of GitHub Repositories. In 2016 IEEE International Conference on Software Maintenance and Evolution (ICSME). IEEE, Washington, DC, USA, 334--344."},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1145\/3180155.3180238"},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1145\/2393596.2393647"},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1145\/1216295.1216309"},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1145\/2950290.2950339"},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1145\/2145204.2145396"},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1017\/S1351324916000243"},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jss.2018.05.065"},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICSME.2016.78"},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1145\/3001896"},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.chb.2017.05.045"},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1145\/2441955.2441970"},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1145\/3134684"},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICIEV.2013.6572650"},{"key":"e_1_2_1_21_1","volume-title":"Towards Designing Cooperative and Social Conversational Agents for Customer Service. In International Conference on Information Systems (ICIS). AIS.","author":"Gnewuch Ulrich","year":"2017","unstructured":"Ulrich Gnewuch, Stefan Morana, and Alexander Maedche. 2017. Towards Designing Cooperative and Social Conversational Agents for Customer Service. In International Conference on Information Systems (ICIS). AIS."},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1145\/2568225.2568260"},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1145\/2884781.2884826"},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.5555\/2818754.2818800"},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1145\/3173574.3174042"},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.5555\/3155562.3155576"},{"key":"e_1_2_1_27_1","volume-title":"Applications and Innovations in Intelligent","author":"Kerry Alice","unstructured":"Alice Kerry, Richard Ellis, and Susan Bull. 2009. Conversational agents in E-Learning. In Applications and Innovations in Intelligent Systems XVI. Springer, London, UK, 169--182."},{"key":"e_1_2_1_28_1","first-page":"1","article-title":"Usability and effectiveness evaluation of a course-advising chat bot","volume":"44","author":"Kim Hyekyung","year":"2007","unstructured":"Hyekyung Kim, Miguel E Ruiz, and Lorna Peterson. 2007. Usability and effectiveness evaluation of a course-advising chat bot. Proceedings of the American Society for Information Science and Technology 44, 1 (2007), 1--5.","journal-title":"Proceedings of the American Society for Information Science and Technology"},{"key":"e_1_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2002.1027796"},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1109\/FUZZY.2010.5584064"},{"key":"e_1_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1109\/MS.2017.4541027"},{"key":"e_1_2_1_32_1","volume-title":"How Software Developers Mitigate Collaboration Friction with Chatbots. CoRR abs\/1702.07011","author":"Lebeuf Carlene","year":"2017","unstructured":"Carlene Lebeuf, Margaret-Anne D. Storey, and Alexey Zagalsky. 2017. How Software Developers Mitigate Collaboration Friction with Chatbots. CoRR abs\/1702.07011 (2017). arXiv:1702.07011 http:\/\/arxiv.org\/abs\/1702.07011"},{"key":"e_1_2_1_33_1","volume-title":"Proceedings of 15th European Conference on Computer-Supported Cooperative Work-Exploratory Papers. EUSSET), 35--54","author":"Lee Minha","year":"2017","unstructured":"Minha Lee, Lily Frank, Femke Beute, Yvonne de Kort, and Wijnand IJsselsteijn. 2017. Bots mind the social-technical gap. In Proceedings of 15th European Conference on Computer-Supported Cooperative Work-Exploratory Papers. EUSSET), 35--54. https:\/\/hdl.handle.net\/20.500.12015\/2929"},{"key":"e_1_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.1145\/2818052.2869117"},{"key":"e_1_2_1_35_1","volume-title":"Intelligent Interactive Multimedia Systems and Services","author":"M\u00e4urer Daniel","unstructured":"Daniel M\u00e4urer and Karsten Weihe. 2015. Benjamin Franklin's decision method is acceptable and helpful with a conversational agent. In Intelligent Interactive Multimedia Systems and Services. Springer, Cham, Switzerland, 109--120."},{"key":"e_1_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.im.2016.11.008"},{"key":"e_1_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.5555\/3155562.3155577"},{"key":"e_1_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-31715-6_45"},{"key":"e_1_2_1_39_1","doi-asserted-by":"publisher","DOI":"10.1145\/3195836.3195859"},{"key":"e_1_2_1_40_1","doi-asserted-by":"publisher","DOI":"10.1145\/3012430.3012625"},{"key":"e_1_2_1_41_1","doi-asserted-by":"publisher","DOI":"10.5555\/3155562.3155652"},{"key":"e_1_2_1_42_1","volume-title":"More Common Than You Think: An In-depth Study of Casual Contributors. In 2016 IEEE 23rd International Conference on Software Analysis, Evolution, and Reengineering (SANER)","volume":"1","author":"Pinto Gustavo","unstructured":"Gustavo Pinto, Igor Steinmacher, and Marco A. Gerosa. 2016. More Common Than You Think: An In-depth Study of Casual Contributors. In 2016 IEEE 23rd International Conference on Software Analysis, Evolution, and Reengineering (SANER), Vol. 1. IEEE Computer Society, Los Alamitos, CA, USA, 112--123."},{"key":"e_1_2_1_43_1","doi-asserted-by":"publisher","DOI":"10.5555\/2486788.2486988"},{"key":"e_1_2_1_44_1","doi-asserted-by":"publisher","DOI":"10.1145\/2889160.2889244"},{"key":"e_1_2_1_45_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0953-5438(02)00029-2"},{"key":"e_1_2_1_46_1","volume-title":"Annual Meeting of the Florida Association of Institutional Research. 1--3.","author":"Romano Jeanine","year":"2006","unstructured":"Jeanine Romano, Jeffrey D. Kromrey, Jesse Coraggio, and Jeff Skowronek. 2006. Appropriate statistics for ordinal level data: Should we really be using t-test and Cohen's d for evaluating group differences on the NSSE and other surveys?. In Annual Meeting of the Florida Association of Institutional Research. 1--3."},{"key":"e_1_2_1_47_1","doi-asserted-by":"publisher","DOI":"10.1145\/2818048.2819985"},{"key":"e_1_2_1_48_1","doi-asserted-by":"publisher","DOI":"10.1109\/CHASE.2013.6614738"},{"key":"e_1_2_1_49_1","unstructured":"Donna Spencer. 2009. Card sorting: Designing usable categories. Rosenfeld Media."},{"key":"e_1_2_1_50_1","volume-title":"Retrieved","author":"Statt Nick","year":"2016","unstructured":"Nick Statt. 2016. Why Google's fancy new AI assistant is just called 'Google'. Retrieved March 21, 2017 from https:\/\/www.theverge.com\/2016\/5\/20\/11721278\/google-ai-assistant-name-vs-alexa-siri. The Verge. Archived from the original on March 21, 2017."},{"key":"e_1_2_1_51_1","doi-asserted-by":"publisher","DOI":"10.1145\/2675133.2675215"},{"key":"e_1_2_1_52_1","doi-asserted-by":"publisher","DOI":"10.1145\/2884781.2884806"},{"key":"e_1_2_1_53_1","doi-asserted-by":"publisher","DOI":"10.1145\/3180155.3180208"},{"key":"e_1_2_1_54_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.infsof.2014.11.001"},{"key":"e_1_2_1_55_1","volume-title":"Let me in: Guidelines for the Successful Onboarding of Newcomers to Open Source Projects","author":"Steinmacher Igor","year":"2018","unstructured":"Igor Steinmacher, Christoph Treude, and Marco Gerosa. 2018. Let me in: Guidelines for the Successful Onboarding of Newcomers to Open Source Projects. IEEE Software Early Access (2018), 1--1."},{"key":"e_1_2_1_56_1","doi-asserted-by":"publisher","DOI":"10.1145\/2950290.2983989"},{"key":"e_1_2_1_57_1","first-page":"491","article-title":"Designing and Evaluating Pedagogic Conversational Agents to Teach Children. World Academy of Science, Engineering and Technology, International Journal of Social, Behavioral, Educational","volume":"11","author":"Tamayo-Moreno Silvia","year":"2017","unstructured":"Silvia Tamayo-Moreno and Diana P\u00e9rez-Mar\u00edn. 2017. Designing and Evaluating Pedagogic Conversational Agents to Teach Children. World Academy of Science, Engineering and Technology, International Journal of Social, Behavioral, Educational, Economic, Business and Industrial Engineering 11, 3 (2017), 491--496.","journal-title":"Economic, Business and Industrial Engineering"},{"key":"e_1_2_1_58_1","first-page":"99","article-title":"Conversational agents improve peer learning through building on prior knowledge","volume":"20","author":"Tegos Stergios","year":"2017","unstructured":"Stergios Tegos and Stavros Demetriadis. 2017. Conversational agents improve peer learning through building on prior knowledge. Educational Technology & Society 20, 1 (2017), 99--111.","journal-title":"Educational Technology & Society"},{"key":"e_1_2_1_59_1","volume-title":"Communications and Informatics (ICACCI), 2016 International Conference on. IEEE","author":"Thomas N T","year":"2016","unstructured":"N T Thomas. 2016. An e-business chatbot using AIML and LSA. In Advances in Computing, Communications and Informatics (ICACCI), 2016 International Conference on. IEEE, Piscataway, NJ, 2740--2742."},{"key":"e_1_2_1_60_1","doi-asserted-by":"publisher","DOI":"10.1145\/2884781.2884800"},{"key":"e_1_2_1_61_1","doi-asserted-by":"publisher","DOI":"10.1145\/2568225.2568315"},{"key":"e_1_2_1_62_1","doi-asserted-by":"publisher","DOI":"10.1145\/2635868.2635882"},{"key":"e_1_2_1_63_1","doi-asserted-by":"publisher","DOI":"10.1093\/mind\/LIX.236.433"},{"key":"e_1_2_1_64_1","doi-asserted-by":"publisher","DOI":"10.1145\/3183519.3183540"},{"key":"e_1_2_1_65_1","doi-asserted-by":"publisher","DOI":"10.1145\/3160504.3160544"},{"key":"e_1_2_1_66_1","doi-asserted-by":"publisher","DOI":"10.1007\/s12559-015-9326-z"},{"key":"e_1_2_1_67_1","doi-asserted-by":"publisher","DOI":"10.1145\/1984642.1984661"},{"key":"e_1_2_1_68_1","volume-title":"Statistical Methods in the Atmospheric Sciences","author":"Wilks Daniel","unstructured":"Daniel Wilks. 2011. Statistical Methods in the Atmospheric Sciences. Academic Press."},{"key":"e_1_2_1_70_1","doi-asserted-by":"publisher","DOI":"10.1109\/SoMeT.2013.6645669"},{"key":"e_1_2_1_71_1","doi-asserted-by":"publisher","DOI":"10.1145\/2556420.2556789"},{"key":"e_1_2_1_72_1","doi-asserted-by":"publisher","DOI":"10.5555\/2820518.2820564"},{"key":"e_1_2_1_73_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.infsof.2016.01.004"},{"key":"e_1_2_1_74_1","doi-asserted-by":"publisher","DOI":"10.5555\/2666719.2666728"},{"key":"e_1_2_1_75_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICSS.2016.16"},{"key":"e_1_2_1_76_1","doi-asserted-by":"publisher","DOI":"10.1109\/5.880078"}],"container-title":["Proceedings of the ACM on Human-Computer Interaction"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3274451","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3274451","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T00:44:36Z","timestamp":1750207476000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3274451"}},"subtitle":["Characterizing and Understanding Bots in OSS Projects"],"short-title":[],"issued":{"date-parts":[[2018,11]]},"references-count":75,"journal-issue":{"issue":"CSCW","published-print":{"date-parts":[[2018,11]]}},"alternative-id":["10.1145\/3274451"],"URL":"https:\/\/doi.org\/10.1145\/3274451","relation":{},"ISSN":["2573-0142"],"issn-type":[{"value":"2573-0142","type":"electronic"}],"subject":[],"published":{"date-parts":[[2018,11]]},"assertion":[{"value":"2018-11-01","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}