{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2022,6,2]],"date-time":"2022-06-02T06:11:29Z","timestamp":1654150289026},"reference-count":29,"publisher":"IGI Global","issue":"1","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2014,1,1]]},"abstract":"<p>Peer reviewing has been touted as a popular instrument to identify good contributions in communities. A problem of peer reviewing is that reviewers have little incentive to make significant effort. To address this problem, the authors introduce a new variant of peer reviewing. It differs from conventional peer reviewing in two ways: First, peers who have made a contribution must also review the contributions made by others. Second, each contributor issues ratings regarding the reviews he has received. To incentivize reviewing, they design an assessment scheme which does not only assess the quality of the contribution made by a peer, but also the quality of the reviews he has submitted. The scheme ranks peers by overall performance, and the ranks determine their payoff. Such a setting gives way to competition among peers. A core challenge however is to elicit objective reviews and ratings. The authors consider two issues which are in the way of this objectiveness: First, they expect preference bias in ratings, i.e., peers tend to prefer reviews with high scores, but dislike reviews with low scores. Second, strategic peers might defame others in their reviews or ratings. This is because they perceive others as competitors. In this paper, they propose a heuristic to address these issues. Further, they carry out a user study in a lecture scenario to evaluate their scheme. It shows that students are incentivized to submit high-quality reviews and that their scheme is effective to evaluate the performance of students.<\/p>","DOI":"10.4018\/ijvcsn.2014010101","type":"journal-article","created":{"date-parts":[[2014,10,1]],"date-time":"2014-10-01T16:10:21Z","timestamp":1412179821000},"page":"1-21","source":"Crossref","is-referenced-by-count":0,"title":["Incentivizing High-Quality Reviews in Peer-to-Peer Settings"],"prefix":"10.4018","volume":"6","author":[{"given":"J.Z.","family":"Yue","sequence":"first","affiliation":[{"name":"Karlsruhe Institute of Technology, Karlsruhe, Germany"}]},{"given":"K.","family":"B\u00f6hm","sequence":"additional","affiliation":[{"name":"Karlsruhe Institute of Technology, Karlsruhe, Germany"}]},{"given":"S.","family":"von Stackelberg","sequence":"additional","affiliation":[{"name":"Karlsruhe Institute of Technology, Karlsruhe, Germany"}]}],"member":"2432","reference":[{"key":"ijvcsn.2014010101-0","doi-asserted-by":"publisher","DOI":"10.1145\/1341531.1341557"},{"key":"ijvcsn.2014010101-1","doi-asserted-by":"publisher","DOI":"10.1177\/001316446002000104"},{"key":"ijvcsn.2014010101-2","doi-asserted-by":"publisher","DOI":"10.1126\/science.7302566"},{"key":"ijvcsn.2014010101-3","unstructured":"Delgado-Marquez, B. L. (2011). Implementation of a European credit transfer system experience in the course of economics with political science students. In Proc. of the Int. Technology,Education and Development Conference (INTED 2011), Valencia, Spain."},{"key":"ijvcsn.2014010101-4","doi-asserted-by":"publisher","DOI":"10.1198\/000313004X775"},{"key":"ijvcsn.2014010101-5","doi-asserted-by":"publisher","DOI":"10.1145\/1809400.1809413"},{"key":"ijvcsn.2014010101-6","doi-asserted-by":"publisher","DOI":"10.1097\/AOG.0b013e31818425d4"},{"key":"ijvcsn.2014010101-7","doi-asserted-by":"publisher","DOI":"10.1136\/bmj.d4797"},{"key":"ijvcsn.2014010101-8","first-page":"1582","article-title":"Reconceiving the tournament of lawyers: Tracking, seeding, and information control in the international labor markets of elite law firms.","volume":"84","author":"M.Gulati","year":"1998","journal-title":"Virginia Law Review"},{"key":"ijvcsn.2014010101-9","doi-asserted-by":"publisher","DOI":"10.1007\/11600930_26"},{"key":"ijvcsn.2014010101-10","doi-asserted-by":"crossref","first-page":"209","DOI":"10.1613\/jair.2621","article-title":"Mechanisms for making crowds truthful.","volume":"34","author":"R.Jurca","year":"2009","journal-title":"Journal of Artificial Intelligence Research"},{"key":"ijvcsn.2014010101-11","doi-asserted-by":"publisher","DOI":"10.1017\/CBO9780511808098.004"},{"key":"ijvcsn.2014010101-12","doi-asserted-by":"crossref","unstructured":"Kaplan, D., Lacetera, N., & Kaplan, C. (2008). Sample size and precision in NIH peer review. Public Library of Science, 3(7).","DOI":"10.1371\/journal.pone.0002761"},{"key":"ijvcsn.2014010101-13","doi-asserted-by":"publisher","DOI":"10.4108\/icst.collaboratecom.2010.33"},{"key":"ijvcsn.2014010101-14","doi-asserted-by":"publisher","DOI":"10.2307\/2529310"},{"key":"ijvcsn.2014010101-15","doi-asserted-by":"publisher","DOI":"10.1086\/261010"},{"issue":"140","key":"ijvcsn.2014010101-16","first-page":"1","article-title":"A technique for the measurement of attitudes.","volume":"22","author":"R.Likert","year":"1932","journal-title":"Archives de Psychologie"},{"key":"ijvcsn.2014010101-17","article-title":"Web-based peer review: The learner as both adapter and reviewer.","volume":"44","author":"E.-F.Liu","year":"2001","journal-title":"IEEE Transactions on Education"},{"key":"ijvcsn.2014010101-18","doi-asserted-by":"publisher","DOI":"10.1037\/0033-2909.102.1.72"},{"key":"ijvcsn.2014010101-19","doi-asserted-by":"publisher","DOI":"10.1287\/mnsc.1050.0379"},{"key":"ijvcsn.2014010101-20","doi-asserted-by":"publisher","DOI":"10.1016\/B978-0-12-590241-0.50006-X"},{"key":"ijvcsn.2014010101-21","doi-asserted-by":"publisher","DOI":"10.1126\/science.1102081"},{"key":"ijvcsn.2014010101-22","doi-asserted-by":"publisher","DOI":"10.1097\/00005650-197201000-00004"},{"key":"ijvcsn.2014010101-23","doi-asserted-by":"publisher","DOI":"10.1177\/016224398501000309"},{"key":"ijvcsn.2014010101-24","doi-asserted-by":"publisher","DOI":"10.1007\/s00355-005-0033-7"},{"key":"ijvcsn.2014010101-25","doi-asserted-by":"crossref","unstructured":"Spector, P. (1992). Summated rating scale construction: An introduction. Sage University Paper series on Quantitative Applications in the Social Sciences.","DOI":"10.4135\/9781412986038"},{"key":"ijvcsn.2014010101-26","doi-asserted-by":"crossref","unstructured":"Sunde, U. (2003). Potential, prizes and performance: Testing tournament theory with professional tennis data. IZA Discussion (paper series nr. 947).","DOI":"10.2139\/ssrn.477442"},{"key":"ijvcsn.2014010101-27","doi-asserted-by":"publisher","DOI":"10.1136\/jech.2006.049817"},{"key":"ijvcsn.2014010101-28","doi-asserted-by":"publisher","DOI":"10.1177\/0272989X9701700108"}],"container-title":["International Journal of Virtual Communities and Social Networking"],"original-title":[],"language":"ng","link":[{"URL":"https:\/\/www.igi-global.com\/viewtitle.aspx?TitleId=116009","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,6,1]],"date-time":"2022-06-01T18:06:46Z","timestamp":1654106806000},"score":1,"resource":{"primary":{"URL":"https:\/\/services.igi-global.com\/resolvedoi\/resolve.aspx?doi=10.4018\/ijvcsn.2014010101"}},"subtitle":["A Feasibility Study with Student Assignments"],"short-title":[],"issued":{"date-parts":[[2014,1,1]]},"references-count":29,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2014,1]]}},"URL":"https:\/\/doi.org\/10.4018\/ijvcsn.2014010101","relation":{},"ISSN":["1942-9010","1942-9029"],"issn-type":[{"value":"1942-9010","type":"print"},{"value":"1942-9029","type":"electronic"}],"subject":[],"published":{"date-parts":[[2014,1,1]]}}}