{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,11,27]],"date-time":"2025-11-27T10:42:55Z","timestamp":1764240175969,"version":"3.41.0"},"reference-count":38,"publisher":"Association for Computing Machinery (ACM)","issue":"CSCW","license":[{"start":{"date-parts":[[2018,11,1]],"date-time":"2018-11-01T00:00:00Z","timestamp":1541030400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/100007249","name":"University of Minnesota","doi-asserted-by":"crossref","id":[{"id":"10.13039\/100007249","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/100011049","name":"Wikimedia Foundation","doi-asserted-by":"crossref","id":[{"id":"10.13039\/100011049","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Proc. ACM Hum.-Comput. Interact."],"published-print":{"date-parts":[[2018,11]]},"abstract":"<jats:p>Bots have been important to peer production's success. Wikipedia, OpenStreetMap, and Wikidata all have taken advantage of automation to perform work at a rate and scale exceeding that of human contributors. Understanding the ways in which humans and bots behave in these communities is an important topic, and one that relies on accurate bot recognition. Yet, in many cases, bot activities are not explicitly flagged and could be mistaken for human contributions. We develop a machine classifier to detect previously unidentified bots using implicit behavioral and other informal editing characteristics. We show that this method yields a high level of fitness under both formal evaluation (PR-AUC: 0.845, ROC-AUC: 0.985) and a qualitative analysis of \"anonymous\" contributor edit sessions. We also show that, in some cases, unflagged bot activities can significantly misrepresent human behavior in analyses. Our model has the potential to support future research and community patrolling activities.<\/jats:p>","DOI":"10.1145\/3274333","type":"journal-article","created":{"date-parts":[[2018,11,1]],"date-time":"2018-11-01T21:21:27Z","timestamp":1541107287000},"page":"1-18","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":15,"title":["Bot Detection in Wikidata Using Behavioral and Other Informal Cues"],"prefix":"10.1145","volume":"2","author":[{"given":"Andrew","family":"Hall","sequence":"first","affiliation":[{"name":"University of Minnesota, Minneapolis, MN, USA"}]},{"given":"Loren","family":"Terveen","sequence":"additional","affiliation":[{"name":"The University of Minnesota, Minneapolis, MN, USA"}]},{"given":"Aaron","family":"Halfaker","sequence":"additional","affiliation":[{"name":"Wikimedia Foundation, San Francisco, CA, USA"}]}],"member":"320","published-online":{"date-parts":[[2018,11]]},"reference":[{"key":"e_1_2_1_1_1","unstructured":"R. Stuart Geiger. 2011. The lives of bots. (2011)."},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1145\/2491055.2491061"},{"key":"e_1_2_1_3_1","doi-asserted-by":"publisher","unstructured":"R. Stuart Geiger and Aaron Halfaker. 2013. Using edit sessions to measure participation in Wikipedia. In CSCW 861--870. 10.1145\/2441776.2441873","DOI":"10.1145\/2441776.2441873"},{"key":"e_1_2_1_4_1","volume-title":"Operationalizing Conflict and Cooperation between Automated Software Agents in Wikipedia: A Replication and Expansion of \"Even Good Bots Fight","author":"Stuart Geiger R.","year":"2017","unstructured":"R. Stuart Geiger and Aaron Halfaker. 2017. Operationalizing Conflict and Cooperation between Automated Software Agents in Wikipedia: A Replication and Expansion of \"Even Good Bots Fight.\" (2017)."},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","unstructured":"Aaron Halfaker Oliver Keyes Daniel Kluver Jacob Thebault-Spieker Tien Nguyen Kenneth Shores Anuradha Uduwage and Morten Warncke-Wang. 2015. User session identification based on strong regularities in inter-activity time. In WWW 410--418. 10.1145\/2736277.2741117","DOI":"10.1145\/2736277.2741117"},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1145\/1641309.1641332"},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1109\/MC.2012.82"},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","unstructured":"Andrew Hall Sarah McRoberts Jacob Thebault-Spieker Yilun Lin Shilad Sen Brent Hecht and Loren Terveen. 2017. Freedom versus standardization: structured data generation in a peer production community. In CHI 6352--6362. 10.1145\/3025453.3025940","DOI":"10.1145\/3025453.3025940"},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.camwa.2012.01.034"},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","unstructured":"Hongwen Kang Kuansan Wang David Soukal Fritz Behr and Zijian Zheng. 2010. Large-scale bot detection for search engines. In GROUP 501--510. 10.1145\/1772690.1772742","DOI":"10.1145\/1772690.1772742"},{"key":"e_1_2_1_11_1","first-page":"19","article-title":"Power of the few vs. wisdom of the crowd: Wikipedia and the rise of the bourgeoisie","volume":"1","author":"Kittur Aniket","year":"2007","unstructured":"Aniket Kittur, Ed Chi, Bryan A. Pendleton, Bongwon Suh, and Todd Mytkowicz. 2007. Power of the few vs. wisdom of the crowd: Wikipedia and the rise of the bourgeoisie. World Wide Web 1, 2 (2007), 19.","journal-title":"World Wide Web"},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.ijhcs.2009.05.004"},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1145\/2788993.2789836"},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1145\/1531674.1531682"},{"key":"e_1_2_1_15_1","doi-asserted-by":"crossref","unstructured":"Alessandro Piscopo Chris Phethean and Elena Simperl. 2017. What Makes a Good Collaborative Knowledge Graph: Group Composition and Quality in Wikidata. In SocInfo 305--322.","DOI":"10.1007\/978-3-319-67217-5_19"},{"key":"e_1_2_1_16_1","volume-title":"1st International Competition on Wikipedia Vandalism Detection. In CLEF (Notebook Papers\/LABs\/Workshops).","author":"Potthast Martin","year":"2010","unstructured":"Martin Potthast, Benno Stein, and Teresa Holfeld. 2010. Overview of the 1st International Competition on Wikipedia Vandalism Detection. In CLEF (Notebook Papers\/LABs\/Workshops)."},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","unstructured":"Amir Sarabadani Aaron Halfaker and Dario Taraborelli. 2017. Building automated vandalism detection tools for Wikidata. In WWW 1647--1654. 10.1145\/3041021.3053366","DOI":"10.1145\/3041021.3053366"},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1145\/2641580.2641613"},{"volume-title":"Intelligent Technologies for Information Analysis","author":"Tan Pang-Ning","key":"e_1_2_1_19_1","unstructured":"Pang-Ning Tan and Vipin Kumar. 2004. Discovery of web robot sessions based on their navigational patterns. In Intelligent Technologies for Information Analysis. Springer, 193--222."},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1145\/1501750.1501770"},{"key":"e_1_2_1_21_1","volume-title":"Even Good Bots Fight. ArXiv Prepr. ArXiv160904285","author":"Tsvetkova Milena","year":"2016","unstructured":"Milena Tsvetkova, Ruth Garc\u00eda-Gavilanes, Luciano Floridi, and Taha Yasseri. 2016. Even Good Bots Fight. ArXiv Prepr. ArXiv160904285 (2016)."},{"key":"e_1_2_1_22_1","volume-title":"ICWSM. Retrieved","author":"Warncke-Wang Morten","year":"2015","unstructured":"Morten Warncke-Wang, Vivek Ranjan, Loren Terveen, and Brent Hecht. 2015. Misalignment Between Supply and Demand of Quality Content in Peer Production Communities. In ICWSM. Retrieved September 16, 2016 from http:\/\/www.aaai.org\/ocs\/index.php\/ICWSM\/ICWSM15\/paper\/view\/10591"},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D17-1213"},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1111\/tgis.12037"},{"key":"e_1_2_1_25_1","unstructured":"2017. Wikipedia:Bot Approvals Group. Wikipedia. Retrieved January 20 2018 from https:\/\/en.wikipedia.org\/w\/index.php?title=Wikipedia:Bot_Approvals_Group&oldid=807843217"},{"key":"e_1_2_1_26_1","unstructured":"2017. Wikipedia:History of Wikipedia bots. Wikipedia. Retrieved January 20 2018 from https:\/\/en.wikipedia.org\/w\/index.php?title=Wikipedia:History_of_Wikipedia_bots&oldid=812914046"},{"key":"e_1_2_1_27_1","unstructured":"2018. Wikipedia:Bot policy. Wikipedia. Retrieved January 20 2018 from https:\/\/en.wikipedia.org\/w\/index.php?title=Wikipedia:Bot_policy&oldid=820435660"},{"volume-title":"Coding (social sciences). Wikipedia. Retrieved","year":"2018","key":"e_1_2_1_28_1","unstructured":"2018. Coding (social sciences). Wikipedia. Retrieved July 8, 2018 from https:\/\/en.wikipedia.org\/w\/index.php?title=Coding_(social_sciences)&oldid=834193623"},{"key":"e_1_2_1_29_1","unstructured":"2018. Wikipedia:AutoWikiBrowser. Wikipedia. Retrieved July 8 2018 from https:\/\/en.wikipedia.org\/w\/index.php?title=Wikipedia:AutoWikiBrowser&oldid=840931199"},{"key":"e_1_2_1_30_1","unstructured":"Wikidata:Bots - Wikidata. Retrieved July 4 2018 from https:\/\/www.wikidata.org\/wiki\/Wikidata:Bots"},{"key":"e_1_2_1_31_1","unstructured":"TIGER - OpenStreetMap Wiki. Retrieved January 20 2018 from https:\/\/wiki.openstreetmap.org\/wiki\/TIGER"},{"key":"e_1_2_1_32_1","unstructured":"TIGER fixup - OpenStreetMap Wiki. Retrieved January 20 2018 from https:\/\/wiki.openstreetmap.org\/wiki\/TIGER_fixup"},{"key":"e_1_2_1_33_1","unstructured":"Import\/Guidelines - OpenStreetMap Wiki. Retrieved January 20 2018 from https:\/\/wiki.openstreetmap.org\/wiki\/Import\/Guidelines"},{"key":"e_1_2_1_34_1","unstructured":"Who Writes Wikipedia? (Aaron Swartz's Raw Thought). Retrieved January 19 2018 from http:\/\/www.aaronsw.com\/weblog\/whowriteswikipedia"},{"key":"e_1_2_1_35_1","unstructured":"Research:Measuring edit productivity - Meta. Retrieved April 17 2018 from https:\/\/meta.wikimedia.org\/wiki\/Research:Measuring_edit_productivity"},{"key":"e_1_2_1_36_1","unstructured":"Wikidata:Glossary - Wikidata. Retrieved July 1 2018 from https:\/\/www.wikidata.org\/wiki\/Wikidata:Glossary"},{"key":"e_1_2_1_37_1","unstructured":"Manual:Tags - MediaWiki. Retrieved July 9 2018 from https:\/\/www.mediawiki.org\/wiki\/Manual:Tags"},{"key":"e_1_2_1_38_1","unstructured":"Proposed features\/changeset tags - OpenStreetMap Wiki. Retrieved September 3 2018 from https:\/\/wiki.openstreetmap.org\/wiki\/Proposed_features\/changeset_tags"}],"container-title":["Proceedings of the ACM on Human-Computer Interaction"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3274333","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3274333","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T00:44:07Z","timestamp":1750207447000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3274333"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2018,11]]},"references-count":38,"journal-issue":{"issue":"CSCW","published-print":{"date-parts":[[2018,11]]}},"alternative-id":["10.1145\/3274333"],"URL":"https:\/\/doi.org\/10.1145\/3274333","relation":{},"ISSN":["2573-0142"],"issn-type":[{"type":"electronic","value":"2573-0142"}],"subject":[],"published":{"date-parts":[[2018,11]]},"assertion":[{"value":"2018-11-01","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}