{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,17]],"date-time":"2025-10-17T14:13:03Z","timestamp":1760710383940,"version":"3.41.0"},"reference-count":115,"publisher":"Association for Computing Machinery (ACM)","issue":"4","license":[{"start":{"date-parts":[[2020,10,16]],"date-time":"2020-10-16T00:00:00Z","timestamp":1602806400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Trans. Soc. Comput."],"published-print":{"date-parts":[[2020,12,31]]},"abstract":"<jats:p>Attempts to explain the success of knowledge co-production communities have focused on organizational design, including structure, motivation, roles, and coordination mechanisms. Meantime, the role that subject-matter-experts play in these knowledge production settings has largely been left in a theoretical and empirical void; its existence has been assumed, but we know little about its nature and scope, as it is difficult to observe. In this article, we start filling that void, using Wikipedia as the setting for our empirical investigation. First, we carefully crossed information from individual Wikipedia editor pages with external sources such as Google Scholar to reliably identify editors who are credentialed experts. Matching these credentialed experts with their Wikipedia editing patterns, we used this dataset to train a machine learning classifier that we then employed to identify additional expert editors and assess the nature and the scope of their work across Wikipedia. Our results suggest that the scope of expert involvement is substantial, albeit with considerable differences across topics. We estimate that approximately 10%--30% of Wikipedia\u2019s contributors have substantial subject-matter expertise in the topics that they edit. We discuss implications for theory and practice of peer-production.<\/jats:p>","DOI":"10.1145\/3416853","type":"journal-article","created":{"date-parts":[[2020,10,16]],"date-time":"2020-10-16T22:30:30Z","timestamp":1602887430000},"page":"1-36","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":6,"title":["Assessing the Contribution of Subject-matter Experts to Wikipedia"],"prefix":"10.1145","volume":"3","author":[{"given":"Alex","family":"Yarovoy","sequence":"first","affiliation":[{"name":"University of Haifa, Mount Carmel, Haifa, Israel"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yiftach","family":"Nagar","sequence":"additional","affiliation":[{"name":"University of Haifa, Mount Carmel, Haifa, Israel"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Einat","family":"Minkov","sequence":"additional","affiliation":[{"name":"University of Haifa, Mount Carmel, Haifa, Israel"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Ofer","family":"Arazy","sequence":"additional","affiliation":[{"name":"University of Haifa, Mount Carmel, Haifa, Israel"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2020,10,16]]},"reference":[{"key":"e_1_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1177\/0170840615584459"},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1145\/290159.290160"},{"volume-title":"Proceedings of the 17th International Conference on World Wide Web. ACM, 665--674","author":"Adamic Lada A.","key":"e_1_2_1_3_1"},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1145\/1242572.1242608"},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1145\/1822258.1822279"},{"volume-title":"The quality of open source production: Zealots and good Samaritans in the case of Wikipedia","year":"2007","author":"Anthony Denise","key":"e_1_2_1_6_1"},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1287\/isre.2016.0647"},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.2753\/MIS0742-1222290303"},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1002\/asi.23579"},{"volume-title":"Proceedings of the 20th Conference on Computer-supported Cooperative Work and Social Computing (CSCW\u201917)","author":"Arazy O.","key":"e_1_2_1_10_1"},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1145\/3366711"},{"volume-title":"forthcoming. The evolutionary trajectories of peer-produced artifacts: Group composition, the trajectories","author":"Arazy Ofer","key":"e_1_2_1_12_1"},{"volume-title":"Proceedings of the 22nd European Conference on Information Systems.","year":"2014","author":"Arazy Ofer","key":"e_1_2_1_13_1"},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.2753\/MIS0742-1222270403"},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1145\/2675133.2675257"},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1002\/asi.22869"},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1145\/2484028.2484183"},{"volume-title":"Online Communities and Governance Mechanisms","author":"Auray N.","key":"e_1_2_1_18_1","doi-asserted-by":"crossref","DOI":"10.1017\/CBO9781139004145.014"},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1097\/MEG.0000000000000003"},{"key":"e_1_2_1_20_1","first-page":"1","article-title":"It was fun, but did it last? The dynamic interplay between fun motives and contributors\u2019 activity in peer-production. Hum.-comput","volume":"1","author":"Balestra Martina","year":"2017","journal-title":"Interact."},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1111\/j.2517-6161.1995.tb02031.x"},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.2307\/1562247"},{"volume-title":"The Wealth of Networks: How Social Production Transforms Markets and Freedom","author":"Benkler Yochai","key":"e_1_2_1_23_1","doi-asserted-by":"crossref","DOI":"10.12987\/9780300127232"},{"volume-title":"Surpassing Ourselves: An Inquiry into the Nature and Implications of Expertise. Open Court.","year":"1993","author":"Bereiter Carl","key":"e_1_2_1_24_1"},{"volume-title":"Proceedings of the International Conference on Weblogs and Social Media (ICWSM\u201908)","author":"Beschastnikh I.","key":"e_1_2_1_25_1"},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1145\/1401890.1401994"},{"volume":"11","volume-title":"Proceedings of the 1st International ExpertFinder Workshop","author":"Breslin John G.","key":"e_1_2_1_27_1"},{"volume-title":"Wikipedia: Jimmy Wales. The Guardian, International edition.","year":"2006","author":"Burkeman Oliver","key":"e_1_2_1_28_1"},{"volume-title":"Wikipedia. In Proceedings of the 26th Annual SIGCHI Conference on Human Factors in Computing Systems (Session: Shared Authoring). ACM, 1101--1110","author":"Butler B.","key":"e_1_2_1_29_1"},{"volume-title":"Kraut","year":"2007","author":"Butler Brian","key":"e_1_2_1_30_1"},{"key":"e_1_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1145\/956863.956965"},{"key":"e_1_2_1_32_1","unstructured":"Nicholas Carr. 2005. The amorality of Web 2.0. [Blog post]. Retrieved from http:\/\/www.roughtype.com\/?p&equals;110. Nicholas Carr. 2005. The amorality of Web 2.0. [Blog post]. Retrieved from http:\/\/www.roughtype.com\/?p&equals;110."},{"volume-title":"Proceedings of the Conference on Recent Advances in Natural Language Processing (RANLP\u201901)","year":"2001","author":"Carreras Xavier","key":"e_1_2_1_33_1"},{"volume-title":"Assessing the accuracy and quality of Wikipedia entries compared to popular online encyclopaedias: A comparative preliminary study across disciplines in English, Spanish and Arabic. Epic","year":"2012","author":"Casebourne Imogen","key":"e_1_2_1_34_1"},{"key":"e_1_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.1145\/2939672.2939785"},{"volume-title":"Wagner","year":"2006","author":"Cianciolo Anna T.","key":"e_1_2_1_36_1"},{"key":"e_1_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.1177\/001316448104100119"},{"volume-title":"Proceedings of the IJCAI\u201903 Workshop on Information Integration on the Web. 73--78","author":"Cohen W. W.","key":"e_1_2_1_38_1"},{"volume-title":"Wikipedia wants more contributions from academics. The Guardian (29","year":"2011","author":"Corbyn Zoe","key":"e_1_2_1_39_1"},{"key":"e_1_2_1_40_1","doi-asserted-by":"publisher","DOI":"10.1145\/1124772.1124928"},{"key":"e_1_2_1_41_1","doi-asserted-by":"publisher","DOI":"10.1145\/1216295.1216309"},{"key":"e_1_2_1_42_1","first-page":"17","article-title":"P@ noptic expert: Searching for experts not just for documents. In Ausweb Poster Proceedings, Queensland","volume":"15","author":"Craswell Nick","year":"2001","journal-title":"Australia"},{"key":"e_1_2_1_43_1","doi-asserted-by":"publisher","DOI":"10.1177\/074171369904900401"},{"volume-title":"Proceedings of the 8th ACM International Conference on Web Search and Data Mining. ACM, 369--378","author":"Dalvi Bhavana","key":"e_1_2_1_44_1"},{"key":"e_1_2_1_45_1","doi-asserted-by":"publisher","DOI":"10.1109\/CIC.2016.044"},{"key":"e_1_2_1_46_1","volume-title":"Proceedings of the 2nd International Conference on Finding Experts on the Web with Semantics","volume":"290","author":"Demartini Gianluca","year":"2007"},{"volume-title":"Machine: The Power of Human Intuition and Expertise in the Era of the Computer","year":"2000","author":"Dreyfus H. L.","key":"e_1_2_1_47_1"},{"key":"e_1_2_1_48_1","doi-asserted-by":"publisher","DOI":"10.1145\/1961209.1961211"},{"volume-title":"An Introduction to the Cambridge Handbook of Expertise and Expert Performance: Its Development, Organization, and Content","author":"Ericsson K. Anders","key":"e_1_2_1_49_1"},{"key":"e_1_2_1_50_1","doi-asserted-by":"publisher","DOI":"10.1037\/0033-295X.100.3.363"},{"volume-title":"Prospects and Limits of the Empirical Study of Expertise: An Introduction","author":"Anders Ericsson K.","key":"e_1_2_1_51_1"},{"key":"e_1_2_1_52_1","doi-asserted-by":"publisher","DOI":"10.1093\/nar\/gkr1195"},{"edition":"2","volume-title":"The Measurement of Interrater Agreement","author":"Fleiss J. L.","key":"e_1_2_1_53_1"},{"key":"e_1_2_1_54_1","doi-asserted-by":"publisher","DOI":"10.2753\/MIS0742-1222260103"},{"volume-title":"Wikipedia editors","year":"2011","author":"Foundation Wikimedia","key":"e_1_2_1_55_1"},{"key":"e_1_2_1_56_1","volume-title":"Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI\u201907)","volume":"7","author":"Gabrilovich Evgeniy","year":"2007"},{"key":"e_1_2_1_57_1","doi-asserted-by":"publisher","DOI":"10.1287\/inte.4.3.28"},{"key":"e_1_2_1_58_1","doi-asserted-by":"publisher","DOI":"10.1093\/nar\/gkq1129"},{"key":"e_1_2_1_59_1","doi-asserted-by":"publisher","DOI":"10.1086\/317671"},{"key":"e_1_2_1_60_1","doi-asserted-by":"publisher","DOI":"10.1038\/438900a"},{"key":"e_1_2_1_61_1","first-page":"157","article-title":"Wikipedia\u2019s labor squeeze and its consequences","volume":"8","author":"Goldman Eric","year":"2010","journal-title":"J. Telecommun. High Technol. Law"},{"key":"e_1_2_1_62_1","doi-asserted-by":"publisher","DOI":"10.25300\/MISQ\/2018\/14084"},{"volume-title":"Patel","year":"1990","author":"Groen Guy J.","key":"e_1_2_1_63_1"},{"key":"e_1_2_1_64_1","doi-asserted-by":"publisher","DOI":"10.1145\/1641309.1641332"},{"key":"e_1_2_1_65_1","doi-asserted-by":"publisher","DOI":"10.5555\/1210555.1210561"},{"volume-title":"Proceedings of the 27th Annual CHI Conference on Human Factors in Computing Systems (CHI\u201909)","author":"Kittur A.","key":"e_1_2_1_66_1"},{"key":"e_1_2_1_67_1","first-page":"3","article-title":"Supervised machine learning: A review of classification techniques","volume":"160","author":"Kotsiantis Sotiris B.","year":"2007","journal-title":"Emerg. Artif. Intell. Applic. Comput. Eng."},{"volume-title":"Proceedings of the Conference on Supporting Group Work. ACM, 167--176","author":"Kriplean T.","key":"e_1_2_1_68_1"},{"key":"e_1_2_1_69_1","volume-title":"Montreal Conference of eTechnologies","volume":"2006","author":"Li Jie","year":"2006"},{"key":"e_1_2_1_70_1","doi-asserted-by":"publisher","DOI":"10.1145\/1985347.1985352"},{"key":"e_1_2_1_71_1","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pcbi.1000941"},{"volume-title":"Proceedings of the Asian Conference on Machine Learning. 269--284","author":"Madani Omid","key":"e_1_2_1_72_1"},{"volume-title":"Unleashing the Crowd: Collaborative Solutions to Wicked Business and Societal Problems","author":"Majchrzak Ann","key":"e_1_2_1_73_1","doi-asserted-by":"crossref","DOI":"10.1007\/978-3-030-25557-2"},{"key":"e_1_2_1_74_1","doi-asserted-by":"publisher","DOI":"10.25300\/MISQ\/2013\/37.2.07"},{"volume-title":"Your Management Style, and Your Life","author":"Malone Thomas W.","key":"e_1_2_1_75_1"},{"volume-title":"Professional Domains, and Expert Roles","author":"Mieg Harald A.","key":"e_1_2_1_77_1"},{"key":"e_1_2_1_78_1","doi-asserted-by":"publisher","DOI":"10.3115\/1614049.1614073"},{"key":"e_1_2_1_79_1","doi-asserted-by":"publisher","DOI":"10.1145\/2492517.2500242"},{"key":"e_1_2_1_80_1","doi-asserted-by":"publisher","DOI":"10.1145\/2145204.2145266"},{"key":"e_1_2_1_81_1","doi-asserted-by":"publisher","DOI":"10.1177\/1461444810365297"},{"key":"e_1_2_1_82_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/W18-2502"},{"key":"e_1_2_1_83_1","doi-asserted-by":"publisher","DOI":"10.22323\/2.09010304"},{"volume-title":"What is Web 2.0: Design patterns and business models for the next generation of software. Communications 8 Strategies 1, 65","year":"2007","author":"O\u2019Reilly Tim","key":"e_1_2_1_84_1"},{"volume-title":"Talking about Machines: An Ethnography of a Modern Job","author":"Orr Julian Edgerton","key":"e_1_2_1_85_1"},{"key":"e_1_2_1_86_1","doi-asserted-by":"publisher","DOI":"10.1145\/2180868.2180872"},{"volume-title":"Groen","year":"1991","author":"Patel Vimla L.","key":"e_1_2_1_87_1"},{"key":"e_1_2_1_88_1","doi-asserted-by":"publisher","DOI":"10.5555\/1953048.2078195"},{"key":"e_1_2_1_89_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10115-014-0794-3"},{"key":"e_1_2_1_90_1","doi-asserted-by":"publisher","DOI":"10.1145\/2939672.2939822"},{"volume-title":"Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics. 742--751","year":"2013","author":"Rokhlenko Oleg","key":"e_1_2_1_91_1"},{"key":"e_1_2_1_92_1","doi-asserted-by":"crossref","unstructured":"Karol G. Ross Jennifer L. Shafer and Gary Klein. 2006. Professional Judgments and \u201cNaturalistic Decision Making.\u201d Karol G. Ross Jennifer L. Shafer and Gary Klein. 2006. Professional Judgments and \u201cNaturalistic Decision Making.\u201d","DOI":"10.1017\/CBO9780511816796.023"},{"key":"e_1_2_1_93_1","unstructured":"Larry Sanger. 2004. Why Wikipedia Must Jettison Its Anti-Elitism [Blog post]. Retrieved from https:\/\/web.archive.org\/web\/20200807044910\/https:\/\/larrysanger.org\/2004\/12\/why-wikipedia-must-jettison-its-anti-elitism\/. Larry Sanger. 2004. Why Wikipedia Must Jettison Its Anti-Elitism [Blog post]. Retrieved from https:\/\/web.archive.org\/web\/20200807044910\/https:\/\/larrysanger.org\/2004\/12\/why-wikipedia-must-jettison-its-anti-elitism\/."},{"key":"e_1_2_1_94_1","doi-asserted-by":"publisher","DOI":"10.3366\/E1742360008000543"},{"key":"e_1_2_1_95_1","doi-asserted-by":"publisher","DOI":"10.1145\/3041021.3053366"},{"key":"e_1_2_1_96_1","doi-asserted-by":"publisher","DOI":"10.1002\/asi.22657"},{"volume-title":"Here Comes Everybody: The Power of Organizing without Organizations","author":"Shirky Clay","key":"e_1_2_1_97_1"},{"key":"e_1_2_1_98_1","doi-asserted-by":"publisher","DOI":"10.1145\/1031171.1031184"},{"key":"e_1_2_1_99_1","doi-asserted-by":"publisher","DOI":"10.1145\/2934687"},{"key":"e_1_2_1_100_1","doi-asserted-by":"publisher","DOI":"10.5555\/1358262.1358274"},{"key":"e_1_2_1_101_1","doi-asserted-by":"publisher","DOI":"10.1108\/00220411111164709"},{"key":"e_1_2_1_102_1","doi-asserted-by":"crossref","DOI":"10.1093\/oso\/9780195189285.001.0001","volume-title":"Infotopia: How Many Minds Produce Knowledge","author":"Sunstein C. R.","year":"2006"},{"volume-title":"Economies, Societies and Nations","author":"Surowiecki J.","key":"e_1_2_1_103_1"},{"volume-title":"Wikinomics: How Mass Collaboration Changes Everything. Portfolio.","year":"2006","author":"Tapscott D.","key":"e_1_2_1_104_1"},{"volume-title":"Wikimania 2011","year":"2011","author":"Taraborelli Dario","key":"e_1_2_1_105_1"},{"key":"e_1_2_1_106_1","first-page":"5238","article-title":"Science is Shaped by Wikipedia: Evidence from a Randomized Control Trial","author":"Thompson Neil","year":"2018","journal-title":"MIT Sloan School Working Paper"},{"volume-title":"Leonardi","year":"2016","author":"Treem Jeffrey W.","key":"e_1_2_1_107_1"},{"key":"e_1_2_1_108_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/W16-2702"},{"volume-title":"Proceedings of the 2nd International Conference on Online Communities and Social Computing (OCSD\u201907)","author":"Viegas F.","key":"e_1_2_1_109_1"},{"volume-title":"Jimmy Wales: The birth of Wikipedia [Video file].","year":"2005","author":"Wales Jimmy","key":"e_1_2_1_110_1"},{"key":"e_1_2_1_111_1","doi-asserted-by":"publisher","DOI":"10.1037\/0022-0663.83.1.73"},{"key":"e_1_2_1_112_1","doi-asserted-by":"publisher","DOI":"10.1111\/j.1558-5646.1989.tb04220.x"},{"volume-title":"Machine Learning and Medical Imaging","author":"Yan J.","key":"e_1_2_1_113_1"},{"volume-title":"Proceedings of the 10th International AAAI Conference on Web and Social Media.","year":"2016","author":"Yang Diyi","key":"e_1_2_1_114_1"},{"key":"e_1_2_1_115_1","doi-asserted-by":"publisher","DOI":"10.1145\/1242572.1242603"},{"volume-title":"Proceedings of the IEEE\/WIC\/ACM International Conference on Web Intelligence. IEEE, 94--100","author":"Zhu Jianhan","key":"e_1_2_1_116_1"}],"container-title":["ACM Transactions on Social Computing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3416853","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3416853","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T22:41:30Z","timestamp":1750200090000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3416853"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,10,16]]},"references-count":115,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2020,12,31]]}},"alternative-id":["10.1145\/3416853"],"URL":"https:\/\/doi.org\/10.1145\/3416853","relation":{},"ISSN":["2469-7818","2469-7826"],"issn-type":[{"type":"print","value":"2469-7818"},{"type":"electronic","value":"2469-7826"}],"subject":[],"published":{"date-parts":[[2020,10,16]]},"assertion":[{"value":"2019-05-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2020-08-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2020-10-16","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}