{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,14]],"date-time":"2026-02-14T04:33:54Z","timestamp":1771043634128,"version":"3.50.1"},"reference-count":66,"publisher":"Association for Computing Machinery (ACM)","issue":"1","license":[{"start":{"date-parts":[[2024,12,28]],"date-time":"2024-12-28T00:00:00Z","timestamp":1735344000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Softw. Eng. Methodol."],"published-print":{"date-parts":[[2025,1,31]]},"abstract":"<jats:p>Stack Overflow incentive system awards users with reputation scores to ensure quality. The decentralized nature of the forum may make the incentive system prone to manipulation. This article offers, for the first time, a comprehensive study of the reported types of reputation manipulation scenarios that might be exercised in Stack Overflow and the prevalence of such reputation gamers by a qualitative study of 1,697 posts from meta Stack Exchange sites. We found four different types of reputation fraud scenarios, such as voting rings where communities form to upvote each other repeatedly on similar posts. We developed algorithms that enable platform managers to automatically identify these suspicious reputation gaming scenarios for review. The first algorithm identifies isolated\/semi-isolated communities where probable reputation frauds may occur mostly by collaborating with each other. The second algorithm looks for sudden unusual big jumps in the reputation scores of users. We evaluated the performance of our algorithms by examining the reputation history dashboard of Stack Overflow users from the Stack Overflow Web site. We observed that around 60\u201380% of users flagged as suspicious by our algorithms experienced reductions in their reputation scores by Stack Overflow.<\/jats:p>","DOI":"10.1145\/3691627","type":"journal-article","created":{"date-parts":[[2024,9,4]],"date-time":"2024-09-04T18:23:09Z","timestamp":1725474189000},"page":"1-41","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":1,"title":["Reputation Gaming in Crowd Technical Knowledge Sharing"],"prefix":"10.1145","volume":"34","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-2263-2984","authenticated-orcid":false,"given":"Iren","family":"Mazloomzadeh","sequence":"first","affiliation":[{"name":"Polytechnique Montr\u00e9al, Montr\u00e9al, QC, Canada"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1376-095X","authenticated-orcid":false,"given":"Gias","family":"Uddin","sequence":"additional","affiliation":[{"name":"York University, Toronto, ON, Canada"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5704-4173","authenticated-orcid":false,"given":"Foutse","family":"Khomh","sequence":"additional","affiliation":[{"name":"Polytechnique Montr\u00e9al, Montr\u00e9al, QC, Canada"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0023-9543","authenticated-orcid":false,"given":"Ashkan","family":"Sami","sequence":"additional","affiliation":[{"name":"Edinburgh Napier University, Edinburgh, U.K."}]}],"member":"320","published-online":{"date-parts":[[2024,12,28]]},"reference":[{"key":"e_1_3_2_2_2","doi-asserted-by":"publisher","DOI":"10.1145\/1341531.1341557"},{"key":"e_1_3_2_3_2","doi-asserted-by":"publisher","DOI":"10.1145\/2339530.2339665"},{"key":"e_1_3_2_4_2","doi-asserted-by":"publisher","DOI":"10.1145\/2488388.2488398"},{"key":"e_1_3_2_5_2","first-page":"1","volume-title":"Proceedings of the ACM CHI Conference on Human Factors in Computing Systems","author":"Antin Judd","year":"2011","unstructured":"Judd Antin and Elizabeth F. Churchill. 2011. Badges in social media: A social psychological perspective. In Proceedings of the ACM CHI Conference on Human Factors in Computing Systems, 1\u20138."},{"key":"e_1_3_2_6_2","doi-asserted-by":"publisher","DOI":"10.1287\/mnsc.1060.0545"},{"key":"e_1_3_2_7_2","doi-asserted-by":"publisher","DOI":"10.1145\/3196398.3196430"},{"key":"e_1_3_2_8_2","first-page":"568","volume-title":"Proceedings of the 27th International Conference on Software Maintenance","author":"Bird Christian","year":"2011","unstructured":"Christian Bird. 2011. Sociotechnical coordination and collaboration in open source software. In Proceedings of the 27th International Conference on Software Maintenance, 568\u2013573."},{"key":"e_1_3_2_9_2","doi-asserted-by":"publisher","DOI":"10.1145\/1453101.1453107"},{"key":"e_1_3_2_10_2","doi-asserted-by":"publisher","DOI":"10.1088\/1742-5468\/2008\/10\/P10008"},{"key":"e_1_3_2_11_2","doi-asserted-by":"publisher","DOI":"10.5555\/3163583.3163673"},{"key":"e_1_3_2_12_2","doi-asserted-by":"publisher","DOI":"10.1109\/MS.2012.169"},{"key":"e_1_3_2_13_2","doi-asserted-by":"publisher","DOI":"10.1145\/2685553.2698999"},{"key":"e_1_3_2_14_2","volume-title":"Designing Reputation Systems for the Social Web","author":"Dellarocas Chrysanthos","year":"2010","unstructured":"Chrysanthos Dellarocas. 2010. Designing Reputation Systems for the Social Web. Technical Report. Boston University School of Management Research Paper 2010-18."},{"key":"e_1_3_2_15_2","doi-asserted-by":"publisher","DOI":"10.1145\/2212877.2212883"},{"key":"e_1_3_2_16_2","first-page":"2425","volume-title":"Proceedings of the 2011 Annual Conference Extended Abstracts on Human Factors in Computing Systems","author":"Deterding Sebastian","year":"2011","unstructured":"Sebastian Deterding, Dan Dixon, Lennart Nacke, Kenton O\u2019Hara, and Miguel Sicart. 2011. Gamification: Using game design elements in non-gaming contexts. In Proceedings of the 2011 Annual Conference Extended Abstracts on Human Factors in Computing Systems, 2425\u20132428."},{"key":"e_1_3_2_17_2","first-page":"273","volume-title":"Proceedings of the 22nd USENIX Security Symposium","author":"Finifter Matthew","year":"2013","unstructured":"Matthew Finifter, Devdatta Akhawe, and David Wagner. 2013. An empirical study of vulnerability rewards programs. In Proceedings of the 22nd USENIX Security Symposium, 273\u2013288."},{"key":"e_1_3_2_18_2","unstructured":"Google BigQuery. 2018. SOTorrent December 2018. Retrieved from https:\/\/bigquery.cloud.google.com\/dataset\/sotorrent-org:2018_12_09"},{"key":"e_1_3_2_19_2","unstructured":"Google BigQuery. 2018. SOTorrent September 2018. Retrieved from https:\/\/bigquery.cloud.google.com\/dataset\/sotorrent-org:2018_09_23"},{"key":"e_1_3_2_20_2","unstructured":"Google BigQuery. 2019. SOTorrent March 2019. Retrieved from https:\/\/bigquery.cloud.google.com\/dataset\/sotorrent-org:2019_03_17"},{"key":"e_1_3_2_21_2","doi-asserted-by":"publisher","DOI":"10.5555\/2487085.2487101"},{"key":"e_1_3_2_22_2","doi-asserted-by":"publisher","DOI":"10.1145\/1357054.1357191"},{"key":"e_1_3_2_23_2","doi-asserted-by":"publisher","DOI":"10.1109\/ESEM.2017.34"},{"key":"e_1_3_2_24_2","first-page":"305","volume-title":"Proceedings of the 28th International Conference on Human Factors in Computing Systems","author":"Hsieh Gary","year":"2010","unstructured":"Gary Hsieh, Robert E. Kraut, and Scott E. Hudson. 2010. Why pay? Exploring how financial incentives are used for question & answer. In Proceedings of the 28th International Conference on Human Factors in Computing Systems, 305\u2013314."},{"key":"e_1_3_2_25_2","first-page":"189","volume-title":"Proceedings of the IEEE Symposium on Visual Languages and Human-Centric Computing","author":"Hudson Nathaniel","year":"2015","unstructured":"Nathaniel Hudson, Parmit K. Chilana, Xiaoyu Guo, Jason Day, and Edmund Liu. 2015. Understanding triggers for clarification requests in community-based software help forums. In Proceedings of the IEEE Symposium on Visual Languages and Human-Centric Computing, 189\u2013193."},{"key":"e_1_3_2_26_2","volume-title":"The Encyclopedia of Human-Computer Interaction","author":"Hudson William","year":"2013","unstructured":"William Hudson. 2013. The Encyclopedia of Human-Computer Interaction (2nd. ed.). The Interaction Design Foundation, Chapter Card Sorting."},{"key":"e_1_3_2_27_2","author":"Jan Steve T. K.","year":"2017","unstructured":"Steve T. K. Jan, Chun Wang, Qing Zhang, and Gang Wang. 2017. Towards Monetary Incentives in Social Q & A Services. Technical Report. arXiv:1703.01333.","journal-title":"Towards Monetary Incentives in Social Q & A Services"},{"key":"e_1_3_2_28_2","first-page":"577","volume-title":"Proceedings of the IEEE International Conference on Software Analysis, Evolution and Reengineering","author":"Kanda Tetsuya","year":"2017","unstructured":"Tetsuya Kanda, Mingyu Guo, Hideaki Hata, and Kenichi Matsumoto. 2017. Towards understanding an open-source bounty: Analysis of Bountysource. In Proceedings of the IEEE International Conference on Software Analysis, Evolution and Reengineering, 577\u2013578."},{"key":"e_1_3_2_29_2","doi-asserted-by":"publisher","DOI":"10.1016\/B978-044452769-1\/50008-1"},{"issue":"6","key":"e_1_3_2_30_2","doi-asserted-by":"crossref","first-page":"923","DOI":"10.1016\/S0048-7333(02)00095-1","article-title":"How open source software works: Free user-to-user assistance","volume":"32","author":"Lakhani Karim R.","year":"2003","unstructured":"Karim R. Lakhani and Eric von Hippel. 2003. How open source software works: Free user-to-user assistance. Journal of Research Policy 32, 6 (2003), 923\u2013943.","journal-title":"Journal of Research Policy"},{"key":"e_1_3_2_31_2","author":"Lee Samuel","year":"2003","unstructured":"Samuel Lee, Nina Moisa, and Marco Wei\ua7b5. 2003. Open Source as a Signalling Device - An Economic Analysis. Technical Report. Working Paper Series: Finance & Accounting 102.","journal-title":"Open Source as a Signalling Device - An Economic Analysis"},{"key":"e_1_3_2_32_2","doi-asserted-by":"publisher","DOI":"10.1145\/2740908.2742129"},{"key":"e_1_3_2_33_2","unstructured":"Bethany Marzewski. 2013. Stack Overflow's 5 Criteria for Hiring Dev Talent. Retrieved from https:\/\/www.smartrecruiters.com\/blog\/stack-overflows-5-criteria-for-hiring-dev-talent\/"},{"key":"e_1_3_2_34_2","unstructured":"Meta StackExchange. 2010. A Number of Duplicates Posted in the Last Hour by Different Users. Retrieved from https:\/\/meta.stackexchange.com\/a\/53807"},{"key":"e_1_3_2_35_2","unstructured":"Meta StackExchange. 2016. Why Does My All Badges and Reputation Lost When I Logged in? [Closed]. Retrieved from https:\/\/meta.stackexchange.com\/a\/272729"},{"key":"e_1_3_2_36_2","unstructured":"Meta StackOverflow. 2016. Voting Rings - How to Handle Organized Groups Upvoting Each Other? Retrieved from https:\/\/meta.stackoverflow.com\/questions\/322114\/voting-rings-how-to-handle-organized-groups-upvoting-each-other#comment339072_322114"},{"key":"e_1_3_2_37_2","doi-asserted-by":"publisher","DOI":"10.1109\/MSR.2019.00074"},{"key":"e_1_3_2_38_2","doi-asserted-by":"publisher","DOI":"10.1145\/2989238.2989239"},{"key":"e_1_3_2_39_2","doi-asserted-by":"publisher","DOI":"10.1109\/MS.2018.2874568"},{"key":"e_1_3_2_40_2","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevE.70.056131"},{"key":"e_1_3_2_41_2","unstructured":"Stack Overflow. 2020 (last accessed). What Is Reputation? How Do I Earn (and Lose) It? Retrieved from https:\/\/stackoverflow.com\/help\/whats-reputation"},{"key":"e_1_3_2_42_2","volume-title":"Crowd Documentation: Exploring the Coverage and the Dynamics of API Discussions on Stack Overflow","author":"Parnin Chris","year":"2012","unstructured":"Chris Parnin, Christoph Treude, Lars Grammel, and Margaret-Anne Storey. 2012. Crowd Documentation: Exploring the Coverage and the Dynamics of API Discussions on Stack Overflow. Technical Report. Georgia Tech."},{"key":"e_1_3_2_43_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10664-015-9397-1"},{"key":"e_1_3_2_44_2","first-page":"343","volume-title":"Proceedings of the 14th International Conference on Quality Software","author":"Ponzanelli Luca","year":"2014","unstructured":"Luca Ponzanelli, Andrea Mocci, Alberto Bacchelli, and Michele Lanza. 2014. Understanding and classifying the quality of technical forum questions. In Proceedings of the 14th International Conference on Quality Software, 343\u2013352."},{"key":"e_1_3_2_45_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSME.2014.90"},{"key":"e_1_3_2_46_2","doi-asserted-by":"publisher","DOI":"10.5555\/2820518.2820578"},{"key":"e_1_3_2_47_2","first-page":"127","volume-title":"Trust Among Strangers in Internet Transactions: Empirical Analysis of eBay's Reputation System","author":"Resnick Paul","year":"2002","unstructured":"Paul Resnick and Richard Zeckhauser. 2002. The economics of the Internet and E-commerce (1st. ed.), Vol. 11. In Trust Among Strangers in Internet Transactions: Empirical Analysis of eBay's Reputation System. Emerald Group Publishing Limited, 127\u2013157."},{"key":"e_1_3_2_48_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10664-010-9150-8"},{"key":"e_1_3_2_49_2","first-page":"479","volume-title":"Proceedings of the 33rd IEEE International Conference on Software Maintenance and Evolution New Idea and Emerging Results","author":"Robillard Martin P.","year":"2017","unstructured":"Martin P. Robillard, Andrian Marcus, Christoph Treude, Gabriele Bavota, Oscar Chaparro, Neil Ernst, Marco Aurelio Gerosa, Michael Godfrey, Michele Lanza, Mario Linares-Vasquez, Gail C. Murphy, Laura Morenox David Shepherd, and Edmund Wong. 2017. On-demand developer documentation. In Proceedings of the 33rd IEEE International Conference on Software Maintenance and Evolution New Idea and Emerging Results, 479\u2013483."},{"issue":"33","key":"e_1_3_2_50_2","first-page":"1","article-title":"An alternative to the hand searching gold standard: Validating methodological search filters using relative recall","volume":"6","author":"Sampson Margaret","year":"2006","unstructured":"Margaret Sampson, Li Zhang, Andra Morrison, Nicholas Barrowman, Tammy J. Clifford, Robert W. Platt, Terry P. Klassen, and David Moher. 2006. An alternative to the hand searching gold standard: Validating methodological search filters using relative recall. BMC Medical Research Methodology 6, 33 (2006), 1\u20139.","journal-title":"BMC Medical Research Methodology"},{"key":"e_1_3_2_51_2","doi-asserted-by":"publisher","DOI":"10.1145\/2568225.2568313"},{"issue":"4","key":"e_1_3_2_52_2","doi-asserted-by":"crossref","first-page":"694","DOI":"10.1109\/TSE.2019.2903039","article-title":"Understanding how and why developers seek and analyze API-related opinions","volume":"47","author":"Uddin Gias","year":"2019","unstructured":"Gias Uddin, Olga Baysal, Latifa Guerrouj, and Foutse Khomh. 2019. Understanding how and why developers seek and analyze API-related opinions. IEEE Transactions on Software Engineering 47, 4 (2019), 694\u2013735.","journal-title":"IEEE Transactions on Software Engineering"},{"key":"e_1_3_2_53_2","doi-asserted-by":"publisher","DOI":"10.5555\/3155562.3155586"},{"key":"e_1_3_2_54_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2019.2900245"},{"key":"e_1_3_2_55_2","doi-asserted-by":"publisher","DOI":"10.1109\/MS.2014.80"},{"key":"e_1_3_2_56_2","doi-asserted-by":"publisher","DOI":"10.1145\/2531602.2531659"},{"key":"e_1_3_2_57_2","doi-asserted-by":"publisher","DOI":"10.1007\/s11257-011-9109-5"},{"key":"e_1_3_2_58_2","first-page":"1","article-title":"How do users revise answers on technical Q & A websites? A case study on stack overflow","volume":"46","author":"Wang Shaowei","year":"2018","unstructured":"Shaowei Wang, Tse-Hsun (Peter) Chen, and Ahmed E. Hassan. 2018. How do users revise answers on technical Q & A websites? A case study on stack overflow. IEEE Transactions on Software Engineering, 46 (2018), 1\u201319.","journal-title":"IEEE Transactions on Software Engineering"},{"key":"e_1_3_2_59_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10664-017-9558-5"},{"key":"e_1_3_2_60_2","doi-asserted-by":"publisher","DOI":"10.5555\/3155562.3155650"},{"key":"e_1_3_2_61_2","author":"Ya Yuan","year":"2013","unstructured":"Yuan Ya, Hanghang Tong, Tao Xie, Leman Akoglu, Feng Xu, and Jian Lu. 2013. Want a Good Answer? Ask a Good Question First! Technical Report. arXiv:1311.6876.","journal-title":"Want a Good Answer? Ask a Good Question First!"},{"issue":"1","key":"e_1_3_2_62_2","first-page":"70","article-title":"Detecting high-quality posts in community question answering sites","volume":"302","author":"Ya Yuan","year":"2015","unstructured":"Yuan Ya, Hanghang Tong, Tao Xie, Leman Akoglu, Feng Xu, and Jian Lu. 2015. Detecting high-quality posts in community question answering sites. Journal of Information Sciences 302, 1 (2015), 70\u201382.","journal-title":"Journal of Information Sciences"},{"key":"e_1_3_2_63_2","doi-asserted-by":"publisher","DOI":"10.25300\/MISQ\/2014\/38.4.05"},{"key":"e_1_3_2_64_2","doi-asserted-by":"publisher","DOI":"10.1145\/3180155.3180260"},{"key":"e_1_3_2_65_2","doi-asserted-by":"publisher","DOI":"10.5325\/jinfopoli.7.2017.0372"},{"key":"e_1_3_2_66_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10664-019-09744-3"},{"key":"e_1_3_2_67_2","author":"Zhou Jiayuan","year":"2019","unstructured":"Jiayuan Zhou, Shaowei Wang, Cor-Paul Bezemer, Ying Zou, and Ahmed E. Hassan. 2019. Bounties in Open Source Development on GitHub: A Case Study of Bountysource Bounties. Technical Report. arXiv:1904.02724.","journal-title":"Bounties in Open Source Development on GitHub: A Case Study of Bountysource Bounties"}],"container-title":["ACM Transactions on Software Engineering and Methodology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3691627","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3691627","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,19]],"date-time":"2025-06-19T01:09:40Z","timestamp":1750295380000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3691627"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,12,28]]},"references-count":66,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2025,1,31]]}},"alternative-id":["10.1145\/3691627"],"URL":"https:\/\/doi.org\/10.1145\/3691627","relation":{},"ISSN":["1049-331X","1557-7392"],"issn-type":[{"value":"1049-331X","type":"print"},{"value":"1557-7392","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,12,28]]},"assertion":[{"value":"2021-07-11","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-07-11","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-12-28","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}