{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,3]],"date-time":"2026-07-03T16:27:21Z","timestamp":1783096041926,"version":"3.54.6"},"reference-count":105,"publisher":"Association for Computing Machinery (ACM)","issue":"CSCW2","license":[{"start":{"date-parts":[[2022,11,7]],"date-time":"2022-11-07T00:00:00Z","timestamp":1667779200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"the Research Grants Council of the Hong Kong Special Administrative Region China under General Research Fund","award":["16203421"],"award-info":[{"award-number":["16203421"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Proc. ACM Hum.-Comput. Interact."],"published-print":{"date-parts":[[2022,11,7]]},"abstract":"<jats:p>People often take user ratings\/reviews into consideration when shopping for products or services online. However, such user-generated data contains self-selection bias that could affect people's decisions and it is hard to resolve this issue completely by algorithms. In this work, we propose to raise people's awareness of the self-selection bias by making three types of information concerning user ratings\/reviews transparent. We distill these three pieces of information, i.e., reviewers' experience, the extremity of emotion, and reported aspect(s), from the definition of self-selection bias and exploration of related literature. We further conduct an online survey to assess people's perceptions of the usefulness of such information and identify the exact facets (e.g., negative emotion) people care about in their decision process. Then, we propose a visual design to make such details behind user reviews transparent and integrate the design into an experimental website for evaluation. The results of a between-subjects study demonstrate that our bias-aware design significantly increases people's awareness of bias and their satisfaction with decision-making. We further offer a series of design implications for improving information transparency and awareness of bias in user-generated content.<\/jats:p>","DOI":"10.1145\/3555597","type":"journal-article","created":{"date-parts":[[2022,11,11]],"date-time":"2022-11-11T22:59:06Z","timestamp":1668207546000},"page":"1-31","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":23,"title":["Bias-Aware Design for Informed Decisions: Raising Awareness of Self-Selection Bias in User Ratings and Reviews"],"prefix":"10.1145","volume":"6","author":[{"given":"Qian","family":"Zhu","sequence":"first","affiliation":[{"name":"The Hong Kong University of Science and Technology, Hong Kong, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Leo Yu-Ho","family":"Lo","sequence":"additional","affiliation":[{"name":"The Hong Kong University of Science and Technology, Hong Kong, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Meng","family":"Xia","sequence":"additional","affiliation":[{"name":"Carnegie Mellon University, Pittsburgh, PA, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Zixin","family":"Chen","sequence":"additional","affiliation":[{"name":"The Hong Kong University of Science and Technology, Hong Kong, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Xiaojuan","family":"Ma","sequence":"additional","affiliation":[{"name":"The Hong Kong University of Science and Technology, Hong Kong, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2022,11,11]]},"reference":[{"key":"e_1_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1111\/joie.12240"},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.geb.2017.11.009"},{"key":"e_1_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1162\/tacl_a_00366"},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1145\/2702613.2732701"},{"key":"e_1_2_1_5_1","first-page":"47","article-title":"The problem with online ratings","volume":"55","author":"Aral Sinan","year":"2014","unstructured":"Sinan Aral . 2014 . The problem with online ratings . MIT Sloan Management Review 55 , 2 (2014), 47 . Sinan Aral. 2014. The problem with online ratings. MIT Sloan Management Review 55, 2 (2014), 47.","journal-title":"MIT Sloan Management Review"},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.dss.2017.03.002"},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1145\/3209581"},{"key":"e_1_2_1_8_1","unstructured":"Elias Bareinboim and Judea Pearl. 2012. Controlling selection bias in causal inference. In Artificial Intelligence and Statistics. PMLR 100--108.  Elias Bareinboim and Judea Pearl. 2012. Controlling selection bias in causal inference. In Artificial Intelligence and Statistics. PMLR 100--108."},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D19-1662"},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1147\/JRD.2019.2942287"},{"key":"e_1_2_1_11_1","first-page":"16","article-title":"Redefining Data Transparency","volume":"52","author":"Bertino Elisa","year":"2019","unstructured":"Elisa Bertino , Shawn Merrill , Alina Nesen , and Christine Utz . 2019 . Redefining Data Transparency : A Multidimensional Approach. Computer 52 , 1 (2019), 16 -- 26 . https:\/\/doi.org\/10.1109\/MC.2018.2890190 10.1109\/MC.2018.2890190 Elisa Bertino, Shawn Merrill, Alina Nesen, and Christine Utz. 2019. Redefining Data Transparency: A Multidimensional Approach. Computer 52, 1 (2019), 16--26. https:\/\/doi.org\/10.1109\/MC.2018.2890190","journal-title":"A Multidimensional Approach. Computer"},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.simpat.2017.05.005"},{"key":"e_1_2_1_13_1","unstructured":"Markus Binder Bernd Heinrich Mathias Klier A. Obermeier and Alexander Schiller. 2019. Explaining the Stars: Aspect-based Sentiment Analysis of Online Customer Reviews. In ECIS.  Markus Binder Bernd Heinrich Mathias Klier A. Obermeier and Alexander Schiller. 2019. Explaining the Stars: Aspect-based Sentiment Analysis of Online Customer Reviews. In ECIS."},{"key":"e_1_2_1_14_1","series-title":"June 20","volume-title":"Amazon changes its key formula for calculating product ratings and displaying reviews. GeekWire","author":"Bishop Todd","unstructured":"Todd Bishop . 2015. Amazon changes its key formula for calculating product ratings and displaying reviews. GeekWire ( June 20 ), http:\/\/www.geekwire.com\/2015\/amazon-changes-its-influential-formula-for-calculating-product-ratings (2015). Todd Bishop. 2015. Amazon changes its key formula for calculating product ratings and displaying reviews. GeekWire (June 20), http:\/\/www.geekwire.com\/2015\/amazon-changes-its-influential-formula-for-calculating-product-ratings (2015)."},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1145\/2428736.2428773"},{"key":"e_1_2_1_16_1","unstructured":"Erik Boiy Pieter Hens Koen Deschacht and Marie-Francine Moens. 2007. Automatic Sentiment Analysis in On-line Text.. In ELPUB. 349--360.  Erik Boiy Pieter Hens Koen Deschacht and Marie-Francine Moens. 2007. Automatic Sentiment Analysis in On-line Text.. In ELPUB. 349--360."},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1191\/1478088706qp063oa"},{"key":"e_1_2_1_18_1","volume-title":"Garnett (Eds.)","volume":"30","author":"Calmon Flavio","year":"2017","unstructured":"Flavio Calmon , Dennis Wei , Bhanukiran Vinzamuri , Karthikeyan Natesan Ramamurthy , and Kush R Varshney . 2017 . Optimized Pre-Processing for Discrimination Prevention. In Advances in Neural Information Processing Systems, I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R . Garnett (Eds.) , Vol. 30 . Curran Associates, Inc. https:\/\/proceedings.neurips.cc\/paper\/ 2017\/file\/9a49a25d845a483fae4be7e341368e36-Paper.pdf Flavio Calmon, Dennis Wei, Bhanukiran Vinzamuri, Karthikeyan Natesan Ramamurthy, and Kush R Varshney. 2017. Optimized Pre-Processing for Discrimination Prevention. In Advances in Neural Information Processing Systems, I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett (Eds.), Vol. 30. Curran Associates, Inc. https:\/\/proceedings.neurips.cc\/paper\/2017\/file\/9a49a25d845a483fae4be7e341368e36-Paper.pdf"},{"key":"e_1_2_1_19_1","volume-title":"Proceedings of the 31st International Conference on Neural Information Processing Systems. 3995--4004","author":"Calmon Flavio P","year":"2017","unstructured":"Flavio P Calmon , Dennis Wei , Bhanukiran Vinzamuri , Karthikeyan Natesan Ramamurthy , and Kush R Varshney . 2017 . Optimized pre-processing for discrimination prevention . In Proceedings of the 31st International Conference on Neural Information Processing Systems. 3995--4004 . Flavio P Calmon, Dennis Wei, Bhanukiran Vinzamuri, Karthikeyan Natesan Ramamurthy, and Kush R Varshney. 2017. Optimized pre-processing for discrimination prevention. In Proceedings of the 31st International Conference on Neural Information Processing Systems. 3995--4004."},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.ijinfomgt.2017.11.001"},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.ijhm.2019.102356"},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICDMW.2015.72"},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1287\/mksc.2018.1090"},{"key":"e_1_2_1_24_1","doi-asserted-by":"crossref","unstructured":"Simona Cicognani Paolo Figini and Marco Magnani. 2016. Social influence bias in online ratings: a field experiment. (2016).  Simona Cicognani Paolo Figini and Marco Magnani. 2016. Social influence bias in online ratings: a field experiment. (2016).","DOI":"10.2139\/ssrn.2737992"},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1145\/3180308.3180366"},{"key":"e_1_2_1_26_1","volume-title":"Lloyd Rutledge, Natalia Stash, Lora Aroyo, and Bob Wielinga.","author":"Cramer Henriette","year":"2008","unstructured":"Henriette Cramer , Vanessa Evers , Satyan Ramlal , Maarten Van Someren , Lloyd Rutledge, Natalia Stash, Lora Aroyo, and Bob Wielinga. 2008 . The effects of transparency on trust in and acceptance of a content-based art recommender. User Modeling and User-adapted interaction 18, 5 (2008), 455. Henriette Cramer, Vanessa Evers, Satyan Ramlal, Maarten Van Someren, Lloyd Rutledge, Natalia Stash, Lora Aroyo, and Bob Wielinga. 2008. The effects of transparency on trust in and acceptance of a content-based art recommender. User Modeling and User-adapted interaction 18, 5 (2008), 455."},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1145\/3460231.3474274"},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1093\/jcr\/ucv047"},{"key":"e_1_2_1_29_1","volume-title":"Impact of online consumer reviews on sales and price strategies: A review and directions for future research. Journal of Product & Brand Management 21 (04","author":"Maeyer Peter De","year":"2012","unstructured":"Peter De Maeyer . 2012. Impact of online consumer reviews on sales and price strategies: A review and directions for future research. Journal of Product & Brand Management 21 (04 2012 ), 132--139. https:\/\/doi.org\/10.1108\/10610421211215599 10.1108\/10610421211215599 Peter De Maeyer. 2012. Impact of online consumer reviews on sales and price strategies: A review and directions for future research. Journal of Product & Brand Management 21 (04 2012), 132--139. https:\/\/doi.org\/10.1108\/10610421211215599"},{"key":"e_1_2_1_30_1","volume-title":"Managing Bias in Human-Annotated Data: Moving Beyond Bias Removal. CoRR abs\/2110.13504","author":"Demartini Gianluca","year":"2021","unstructured":"Gianluca Demartini , Kevin Roitero , and Stefano Mizzaro . 2021. Managing Bias in Human-Annotated Data: Moving Beyond Bias Removal. CoRR abs\/2110.13504 ( 2021 ). arXiv:2110.13504 https:\/\/arxiv.org\/abs\/2110.13504 Gianluca Demartini, Kevin Roitero, and Stefano Mizzaro. 2021. Managing Bias in Human-Annotated Data: Moving Beyond Bias Removal. CoRR abs\/2110.13504 (2021). arXiv:2110.13504 https:\/\/arxiv.org\/abs\/2110.13504"},{"key":"e_1_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1145\/2844110"},{"key":"e_1_2_1_32_1","volume-title":"Bootstrap confidence intervals. Statistical science 11, 3","author":"DiCiccio Thomas J","year":"1996","unstructured":"Thomas J DiCiccio and Bradley Efron . 1996. Bootstrap confidence intervals. Statistical science 11, 3 ( 1996 ), 189--228. Thomas J DiCiccio and Bradley Efron. 1996. Bootstrap confidence intervals. Statistical science 11, 3 (1996), 189--228."},{"key":"e_1_2_1_33_1","volume-title":"Advanced Review Helpfulness Modeling. Ph. D. Dissertation","author":"Jiahua Du.","unstructured":"Jiahua Du. 2020. Advanced Review Helpfulness Modeling. Ph. D. Dissertation . Victoria University . Jiahua Du. 2020. Advanced Review Helpfulness Modeling. Ph. D. Dissertation. Victoria University."},{"key":"e_1_2_1_34_1","doi-asserted-by":"crossref","unstructured":"Nico Ebert Kurt A. Ackermann and Bjorn Scheppler. 2021. Bolder is Better: Raising User Awareness through Salient and Concise Privacy Notices. (2021).  Nico Ebert Kurt A. Ackermann and Bjorn Scheppler. 2021. Bolder is Better: Raising User Awareness through Salient and Concise Privacy Notices. (2021).","DOI":"10.1145\/3411764.3445516"},{"key":"e_1_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.1609\/icwsm.v11i1.14898"},{"key":"e_1_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.1145\/3290605.3300724"},{"key":"e_1_2_1_37_1","volume-title":"Frankish","author":"Evans Jonathan St BT","year":"2009","unstructured":"Jonathan St BT Evans and Keith Ed Frankish . 2009 . In two minds: Dual processes and beyond. Oxford University Press . Jonathan St BT Evans and Keith Ed Frankish. 2009. In two minds: Dual processes and beyond. Oxford University Press."},{"key":"e_1_2_1_38_1","volume-title":"Zettlemoyer","author":"Gardner Matt","year":"2017","unstructured":"Matt Gardner , Joel Grus , Mark Neumann , Oyvind Tafjord , Pradeep Dasigi , Nelson F. Liu , Matthew Peters , Michael Schmitz , and Luke S . Zettlemoyer . 2017 . AllenNLP: A Deep Semantic Natural Language Processing Platform . arXiv:arXiv:1803.07640 Matt Gardner, Joel Grus, Mark Neumann, Oyvind Tafjord, Pradeep Dasigi, Nelson F. Liu, Matthew Peters, Michael Schmitz, and Luke S. Zettlemoyer. 2017. AllenNLP: A Deep Semantic Natural Language Processing Platform. arXiv:arXiv:1803.07640"},{"key":"e_1_2_1_39_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.ijhcs.2013.12.007"},{"key":"e_1_2_1_40_1","doi-asserted-by":"publisher","DOI":"10.5555\/1038262.1038777"},{"key":"e_1_2_1_41_1","volume-title":"Proceedings of the International AAAI Conference on Web and Social Media","volume":"9","author":"Gong Wei","year":"2015","unstructured":"Wei Gong , Ee-Peng Lim , and Feida Zhu . 2015 . Characterizing silent users in social media communities . In Proceedings of the International AAAI Conference on Web and Social Media , Vol. 9 . Wei Gong, Ee-Peng Lim, and Feida Zhu. 2015. Characterizing silent users in social media communities. In Proceedings of the International AAAI Conference on Web and Social Media, Vol. 9."},{"key":"e_1_2_1_42_1","unstructured":"Google. 2009. Google dashboard. (2009). https:\/\/myaccount.google.com\/dashboard  Google. 2009. Google dashboard. (2009). https:\/\/myaccount.google.com\/dashboard"},{"key":"#cr-split#-e_1_2_1_43_1.1","unstructured":"Maarten Grootendorst. 2020. KeyBERT: Minimal keyword extraction with BERT. https:\/\/doi.org\/10.5281\/zenodo.4461265 10.5281\/zenodo.4461265"},{"key":"#cr-split#-e_1_2_1_43_1.2","unstructured":"Maarten Grootendorst. 2020. KeyBERT: Minimal keyword extraction with BERT. https:\/\/doi.org\/10.5281\/zenodo.4461265"},{"key":"e_1_2_1_44_1","first-page":"6","article-title":"The Ubiquity of Subjectivity","volume":"42","author":"Halevy Alon Y","year":"2019","unstructured":"Alon Y Halevy . 2019 . The Ubiquity of Subjectivity . IEEE Data Eng. Bull. 42 , 1 (2019), 6 -- 9 . Alon Y Halevy. 2019. The Ubiquity of Subjectivity. IEEE Data Eng. Bull. 42, 1 (2019), 6--9.","journal-title":"IEEE Data Eng. Bull."},{"key":"e_1_2_1_45_1","doi-asserted-by":"publisher","DOI":"10.1108\/OIR-07-2016-0201"},{"key":"e_1_2_1_46_1","volume-title":"Proceedings of the CSCW 2021 Workshop--Investigating and Mitigating Biases in Crowdsourced Data. arXiv preprint arXiv:2111","author":"Hettiachchi Danula","year":"2021","unstructured":"Danula Hettiachchi , Mark Sanderson , Jorge Goncalves , Simo Hosio , Gabriella Kazai , Matthew Lease , Mike Schaekermann , and Emine Yilmaz . 2021 . Proceedings of the CSCW 2021 Workshop--Investigating and Mitigating Biases in Crowdsourced Data. arXiv preprint arXiv:2111 .14322 (2021). Danula Hettiachchi, Mark Sanderson, Jorge Goncalves, Simo Hosio, Gabriella Kazai, Matthew Lease, Mike Schaekermann, and Emine Yilmaz. 2021. Proceedings of the CSCW 2021 Workshop--Investigating and Mitigating Biases in Crowdsourced Data. arXiv preprint arXiv:2111.14322 (2021)."},{"key":"e_1_2_1_47_1","volume-title":"Available at: http:\/\/fivethirtyeight. com\/features\/fandango-movies-ratings","author":"Hickey Walt","year":"2015","unstructured":"Walt Hickey . 2015. Be suspicious of online movie ratings, especially Fandango's. FiveThirtyEight , Available at: http:\/\/fivethirtyeight. com\/features\/fandango-movies-ratings ( 2015 ). Walt Hickey. 2015. Be suspicious of online movie ratings, especially Fandango's. FiveThirtyEight, Available at: http:\/\/fivethirtyeight. com\/features\/fandango-movies-ratings (2015)."},{"key":"e_1_2_1_48_1","volume-title":"Overcoming the J-Shaped Distribution of Product Reviews","author":"Hu Nan","unstructured":"Nan Hu , Paul A. Pavlou , and Jie (Jennifer) Zhang . 2009. Overcoming the J-Shaped Distribution of Product Reviews , Vol. 52 . Communications of the ACM. Nan Hu, Paul A. Pavlou, and Jie (Jennifer) Zhang. 2009. Overcoming the J-Shaped Distribution of Product Reviews, Vol. 52. Communications of the ACM."},{"key":"e_1_2_1_49_1","doi-asserted-by":"publisher","DOI":"10.1145\/1562764.1562800"},{"key":"e_1_2_1_50_1","doi-asserted-by":"publisher","DOI":"10.1109\/STAST.2013.11"},{"key":"e_1_2_1_51_1","doi-asserted-by":"publisher","DOI":"10.1145\/3479859"},{"key":"e_1_2_1_52_1","doi-asserted-by":"publisher","DOI":"10.1145\/3018661.3018699"},{"key":"e_1_2_1_53_1","volume-title":"fast and slow","author":"Kahneman Daniel","unstructured":"Daniel Kahneman . 2011. Thinking , fast and slow . Macmillan . Daniel Kahneman. 2011. Thinking, fast and slow. Macmillan."},{"key":"e_1_2_1_54_1","volume-title":"Nordic Conference on Secure IT Systems. Springer, 131--148","author":"Kani-Zabihi Elahe","year":"2011","unstructured":"Elahe Kani-Zabihi and Martin Helmhout . 2011 . Increasing service users' privacy awareness by introducing on-line interactive privacy features . In Nordic Conference on Secure IT Systems. Springer, 131--148 . Elahe Kani-Zabihi and Martin Helmhout. 2011. Increasing service users' privacy awareness by introducing on-line interactive privacy features. In Nordic Conference on Secure IT Systems. Springer, 131--148."},{"key":"e_1_2_1_55_1","doi-asserted-by":"crossref","unstructured":"H\u00fclya Karaman. 0. Online Review Solicitations Reduce Extremity Bias in Online Review Distributions and Increase Their Representativeness. Management Science 0 0 (0) null. https:\/\/doi.org\/10.1287\/mnsc.2020.3758    10.1287\/mnsc.2020.3758\nH\u00fclya Karaman. 0. Online Review Solicitations Reduce Extremity Bias in Online Review Distributions and Increase Their Representativeness. Management Science 0 0 (0) null. https:\/\/doi.org\/10.1287\/mnsc.2020.3758","DOI":"10.1287\/mnsc.2020.3758"},{"key":"e_1_2_1_56_1","doi-asserted-by":"publisher","DOI":"10.1109\/ARES.2010.51"},{"key":"e_1_2_1_57_1","doi-asserted-by":"publisher","DOI":"10.1145\/3411764.3445444"},{"key":"e_1_2_1_58_1","volume-title":"Ka Leong Daniel Chong, and Nicholas Chan Khin Whai","author":"Hoong Lee Angela Siew","year":"2019","unstructured":"Angela Siew Hoong Lee , Ka Leong Daniel Chong, and Nicholas Chan Khin Whai . 2019 . OpinionSeer: Text Visualization on Hotel Customer Reviews of Services and Physical Environment. In Information Science and Applications 2018, Kuinam J. Kim and Nakhoon Baek (Eds.). Springer Singapore , Singapore, 337--349. Angela Siew Hoong Lee, Ka Leong Daniel Chong, and Nicholas Chan Khin Whai. 2019. OpinionSeer: Text Visualization on Hotel Customer Reviews of Services and Physical Environment. In Information Science and Applications 2018, Kuinam J. Kim and Nakhoon Baek (Eds.). Springer Singapore, Singapore, 337--349."},{"key":"e_1_2_1_59_1","doi-asserted-by":"publisher","DOI":"10.1287\/isre.1070.0154"},{"key":"e_1_2_1_60_1","volume-title":"Jinfeng Li, Saran Mumick, Alon Halevy, Vivian Li, and Wang-Chiew Tan.","author":"Li Yuliang","year":"2019","unstructured":"Yuliang Li , Aaron Xixuan Feng , Jinfeng Li, Saran Mumick, Alon Halevy, Vivian Li, and Wang-Chiew Tan. 2019 . Subjective databases. arXiv preprint arXiv:1902.09661 (2019). Yuliang Li, Aaron Xixuan Feng, Jinfeng Li, Saran Mumick, Alon Halevy, Vivian Li, and Wang-Chiew Tan. 2019. Subjective databases. arXiv preprint arXiv:1902.09661 (2019)."},{"key":"e_1_2_1_61_1","doi-asserted-by":"publisher","DOI":"10.1115\/1.4037612"},{"key":"e_1_2_1_62_1","doi-asserted-by":"publisher","DOI":"10.1080\/19368620903235803"},{"key":"e_1_2_1_63_1","volume-title":"Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692","author":"Liu Yinhan","year":"2019","unstructured":"Yinhan Liu , Myle Ott , Naman Goyal , Jingfei Du , Mandar Joshi , Danqi Chen , Omer Levy , Mike Lewis , Luke Zettlemoyer , and Veselin Stoyanov . 2019 . Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692 (2019). Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692 (2019)."},{"key":"e_1_2_1_64_1","doi-asserted-by":"publisher","DOI":"10.1257\/aer.104.8.2421"},{"key":"e_1_2_1_65_1","doi-asserted-by":"publisher","DOI":"10.1257\/aer.104.8.2421"},{"key":"e_1_2_1_66_1","volume-title":"The Corsini encyclopedia of psychology","author":"McKnight Patrick E","year":"2010","unstructured":"Patrick E McKnight and Julius Najab . 2010. Mann-Whitney U Test . The Corsini encyclopedia of psychology ( 2010 ), 1--1. Patrick E McKnight and Julius Najab. 2010. Mann-Whitney U Test. The Corsini encyclopedia of psychology (2010), 1--1."},{"key":"e_1_2_1_67_1","unstructured":"Mozilla. 2013. Lightbeam add-on for Firefox. (2013). https:\/\/www.mozilla.org\/en-US\/lightbeam\/  Mozilla. 2013. Lightbeam add-on for Firefox. (2013). https:\/\/www.mozilla.org\/en-US\/lightbeam\/"},{"key":"e_1_2_1_68_1","volume-title":"Visualization analysis and design","author":"Munzner Tamara","unstructured":"Tamara Munzner . 2014. Visualization analysis and design . CRC press . Tamara Munzner. 2014. Visualization analysis and design. CRC press."},{"key":"e_1_2_1_69_1","volume-title":"Local Customer Review Survey. Bright Ideas\/Research (12","author":"Murphy Rosie","year":"2020","unstructured":"Rosie Murphy . 2020. Local Customer Review Survey. Bright Ideas\/Research (12 2020 ). https:\/\/www.brightlocal.com\/research\/local-consumer-review-survey\/ Rosie Murphy. 2020. Local Customer Review Survey. Bright Ideas\/Research (12 2020). https:\/\/www.brightlocal.com\/research\/local-consumer-review-survey\/"},{"key":"e_1_2_1_70_1","doi-asserted-by":"publisher","DOI":"10.1111\/puar.13211"},{"key":"e_1_2_1_71_1","doi-asserted-by":"publisher","DOI":"10.1111\/puar.13211"},{"key":"e_1_2_1_72_1","volume-title":"Lumos: Increasing awareness of analytic behavior during visual data analysis","author":"Narechania Arpit","year":"2021","unstructured":"Arpit Narechania , Adam Coscia , Emily Wall , and Alex Endert . 2021 . Lumos: Increasing awareness of analytic behavior during visual data analysis . IEEE Transactions on Visualization and Computer Graphics ( 2021). Arpit Narechania, Adam Coscia, Emily Wall, and Alex Endert. 2021. Lumos: Increasing awareness of analytic behavior during visual data analysis. IEEE Transactions on Visualization and Computer Graphics (2021)."},{"key":"e_1_2_1_73_1","doi-asserted-by":"publisher","DOI":"10.1177\/0146167204271418"},{"key":"e_1_2_1_74_1","volume-title":"Uniwalk: Explainable and accurate recommendation for rating and network data. arXiv preprint arXiv:1710.07134","author":"Park Haekyu","year":"2017","unstructured":"Haekyu Park , Hyunsik Jeon , Junghwan Kim , Beunguk Ahn , and U Kang . 2017 . Uniwalk: Explainable and accurate recommendation for rating and network data. arXiv preprint arXiv:1710.07134 (2017). Haekyu Park, Hyunsik Jeon, Junghwan Kim, Beunguk Ahn, and U Kang. 2017. Uniwalk: Explainable and accurate recommendation for rating and network data. arXiv preprint arXiv:1710.07134 (2017)."},{"key":"e_1_2_1_75_1","doi-asserted-by":"publisher","DOI":"10.1145\/3290605.3300474"},{"key":"e_1_2_1_76_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jesp.2017.01.006"},{"key":"e_1_2_1_77_1","volume-title":"10th Symposium On Usable Privacy and Security ({SOUPS}","author":"Rader Emilee","year":"2014","unstructured":"Emilee Rader . 2014 . Awareness of behavioral tracking and information privacy concern in facebook and google . In 10th Symposium On Usable Privacy and Security ({SOUPS} 2014). 51--67. Emilee Rader. 2014. Awareness of behavioral tracking and information privacy concern in facebook and google. In 10th Symposium On Usable Privacy and Security ({SOUPS} 2014). 51--67."},{"key":"e_1_2_1_78_1","doi-asserted-by":"publisher","DOI":"10.1145\/3173574.3173677"},{"key":"e_1_2_1_79_1","volume-title":"international conference on machine learning. PMLR, 1670--1679","author":"Schnabel Tobias","year":"2016","unstructured":"Tobias Schnabel , Adith Swaminathan , Ashudeep Singh , Navin Chandak , and Thorsten Joachims . 2016 . Recommendations as treatments: Debiasing learning and evaluation . In international conference on machine learning. PMLR, 1670--1679 . Tobias Schnabel, Adith Swaminathan, Ashudeep Singh, Navin Chandak, and Thorsten Joachims. 2016. Recommendations as treatments: Debiasing learning and evaluation. In international conference on machine learning. PMLR, 1670--1679."},{"key":"e_1_2_1_80_1","volume-title":"The Extreme Distribution of Online Reviews: Prevalence, Drivers and Implications. SSRN Electronic Journal (01","author":"Schoenmueller Verena","year":"2018","unstructured":"Verena Schoenmueller , Oded Netzer , and Florian Stahl . 2018. The Extreme Distribution of Online Reviews: Prevalence, Drivers and Implications. SSRN Electronic Journal (01 2018 ). https:\/\/doi.org\/10.2139\/ssrn.3100217 10.2139\/ssrn.3100217 Verena Schoenmueller, Oded Netzer, and Florian Stahl. 2018. The Extreme Distribution of Online Reviews: Prevalence, Drivers and Implications. SSRN Electronic Journal (01 2018). https:\/\/doi.org\/10.2139\/ssrn.3100217"},{"key":"e_1_2_1_81_1","volume-title":"The extreme distribution of online reviews: Prevalence, drivers and implications. Columbia Business School Research Paper 18--10","author":"Schoenm\u00fcller Verena","year":"2019","unstructured":"Verena Schoenm\u00fcller , Oded Netzer , and Florian Stahl . 2019. The extreme distribution of online reviews: Prevalence, drivers and implications. Columbia Business School Research Paper 18--10 ( 2019 ). Verena Schoenm\u00fcller, Oded Netzer, and Florian Stahl. 2019. The extreme distribution of online reviews: Prevalence, drivers and implications. Columbia Business School Research Paper 18--10 (2019)."},{"key":"e_1_2_1_82_1","volume-title":"Estimating sequential bias in online reviews: A Kalman filtering approach. Knowledge Based Systems - KBS 27 (01","author":"Sikora Riyaz","year":"2011","unstructured":"Riyaz Sikora and Kriti Chauhan . 2011. Estimating sequential bias in online reviews: A Kalman filtering approach. Knowledge Based Systems - KBS 27 (01 2011 ). https:\/\/doi.org\/10.1016\/j.knosys.2011.10.011 10.1016\/j.knosys.2011.10.011 Riyaz Sikora and Kriti Chauhan. 2011. Estimating sequential bias in online reviews: A Kalman filtering approach. Knowledge Based Systems - KBS 27 (01 2011). https:\/\/doi.org\/10.1016\/j.knosys.2011.10.011"},{"key":"e_1_2_1_83_1","volume-title":"Proceedings of the 2013 conference on empirical methods in natural language processing. 1631--1642","author":"Socher Richard","year":"2013","unstructured":"Richard Socher , Alex Perelygin , Jean Wu , Jason Chuang , Christopher D Manning , Andrew Y Ng , and Christopher Potts . 2013 . Recursive deep models for semantic compositionality over a sentiment treebank . In Proceedings of the 2013 conference on empirical methods in natural language processing. 1631--1642 . Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D Manning, Andrew Y Ng, and Christopher Potts. 2013. Recursive deep models for semantic compositionality over a sentiment treebank. In Proceedings of the 2013 conference on empirical methods in natural language processing. 1631--1642."},{"key":"e_1_2_1_84_1","volume-title":"Awareness Surveys: The Data-Driven Way to Read People's Minds. Human of Data","author":"Srivastava Anumeha","year":"2016","unstructured":"Anumeha Srivastava . 2016 . Awareness Surveys: The Data-Driven Way to Read People's Minds. Human of Data (2016). Anumeha Srivastava. 2016. Awareness Surveys: The Data-Driven Way to Read People's Minds. Human of Data (2016)."},{"key":"e_1_2_1_85_1","doi-asserted-by":"crossref","unstructured":"Jonathan AC Sterne Matthias Egger and David Moher. 2008. Addressing reporting biases. Cochrane handbook for systematic reviews of interventions: Cochrane book series (2008) 297--333.  Jonathan AC Sterne Matthias Egger and David Moher. 2008. Addressing reporting biases. Cochrane handbook for systematic reviews of interventions: Cochrane book series (2008) 297--333.","DOI":"10.1002\/9780470712184.ch10"},{"key":"e_1_2_1_86_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.bushor.2018.01.007"},{"key":"e_1_2_1_87_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.acl-main.513"},{"key":"e_1_2_1_88_1","doi-asserted-by":"publisher","DOI":"10.1145\/3134736"},{"key":"e_1_2_1_89_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.tourman.2020.104122"},{"key":"e_1_2_1_90_1","unstructured":"C. Walker and Scott Buttinger. 2017. Towards Mitigating Bias in Online Reviews : An Application to Amazon.  C. Walker and Scott Buttinger. 2017. Towards Mitigating Bias in Online Reviews : An Application to Amazon."},{"key":"e_1_2_1_91_1","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2021.3114862"},{"key":"e_1_2_1_92_1","doi-asserted-by":"publisher","DOI":"10.2753\/MIS0742-1222230410"},{"key":"e_1_2_1_93_1","volume-title":"ExtremeReader: An Interactive Explorer for Customizable and Explainable Review Summarization (WWW '20)","author":"Wang Xiaolan","unstructured":"Xiaolan Wang , Yoshihiko Suhara , Natalie Nuno , Yuliang Li , Jinfeng Li , Nofar Carmeli , Stefanos Angelidis , Eser Kandogann , and Wang-Chiew Tan . 2020. ExtremeReader: An Interactive Explorer for Customizable and Explainable Review Summarization (WWW '20) . Association for Computing Machinery , New York, NY, USA , 176--180. https:\/\/doi.org\/10.1145\/3366424.3383535 10.1145\/3366424.3383535 Xiaolan Wang, Yoshihiko Suhara, Natalie Nuno, Yuliang Li, Jinfeng Li, Nofar Carmeli, Stefanos Angelidis, Eser Kandogann, and Wang-Chiew Tan. 2020. ExtremeReader: An Interactive Explorer for Customizable and Explainable Review Summarization (WWW '20). Association for Computing Machinery, New York, NY, USA, 176--180. https:\/\/doi.org\/10.1145\/3366424.3383535"},{"key":"e_1_2_1_94_1","unstructured":"Ding Wu Xunhua Guo and Guoqing Chen. 2017. Mitigating the Dependence Bias in Online Ratings: A ?Consider-the-Opposite\" Strategy for Scale Prompting. (2017).  Ding Wu Xunhua Guo and Guoqing Chen. 2017. Mitigating the Dependence Bias in Online Ratings: A ?Consider-the-Opposite\" Strategy for Scale Prompting. (2017)."},{"key":"e_1_2_1_95_1","unstructured":"Ding Wu Xunhua Guo and Guoqing Chen. 2017. Mitigating the Dependence Bias in Online Ratings: A ?Consider-the-Opposite\" Strategy for Scale Prompting. (2017).  Ding Wu Xunhua Guo and Guoqing Chen. 2017. Mitigating the Dependence Bias in Online Ratings: A ?Consider-the-Opposite\" Strategy for Scale Prompting. (2017)."},{"key":"e_1_2_1_96_1","article-title":"OpinionSeer: Interactive Visualization of Hotel Customer Feedback","author":"Wu Yingcai","year":"2010","unstructured":"Yingcai Wu , Furu Wei , Shixia Liu , Norman Au , Weiwei Cui , Hong Zhou , and Huamin Qu . 2010 . OpinionSeer: Interactive Visualization of Hotel Customer Feedback . IEEE Transactions on Visualization and Computer Graphics ( November 2010), 1109--1118. https:\/\/www.microsoft.com\/en-us\/research\/publication\/opinionseer-interactive-visualization-hotel-customer-feedback\/ Yingcai Wu, Furu Wei, Shixia Liu, Norman Au, Weiwei Cui, Hong Zhou, and Huamin Qu. 2010. OpinionSeer: Interactive Visualization of Hotel Customer Feedback. IEEE Transactions on Visualization and Computer Graphics (November 2010), 1109--1118. https:\/\/www.microsoft.com\/en-us\/research\/publication\/opinionseer-interactive-visualization-hotel-customer-feedback\/","journal-title":"IEEE Transactions on Visualization and Computer Graphics"},{"key":"e_1_2_1_97_1","volume-title":"Ameneh Gholipour Shahraki, and Osmar R. Zaiane","author":"Yadollahi Ali","year":"2017","unstructured":"Ali Yadollahi , Ameneh Gholipour Shahraki, and Osmar R. Zaiane . 2017 . Current State of Text Sentiment Analysis from Opinion to Emotion Mining. ACM Comput. Surv . 50, 2 (2017). https:\/\/doi.org\/10.1145\/3057270 10.1145\/3057270 Ali Yadollahi, Ameneh Gholipour Shahraki, and Osmar R. Zaiane. 2017. Current State of Text Sentiment Analysis from Opinion to Emotion Mining. ACM Comput. Surv. 50, 2 (2017). https:\/\/doi.org\/10.1145\/3057270"},{"key":"e_1_2_1_98_1","doi-asserted-by":"publisher","DOI":"10.1109\/WI-IAT.2010.131"},{"key":"e_1_2_1_99_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-39345-7_39"},{"key":"e_1_2_1_100_1","doi-asserted-by":"publisher","DOI":"10.1145\/3313831.3376235"},{"key":"e_1_2_1_101_1","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3362651","article-title":"Understanding assimilation-contrast effects in online rating systems: modelling, debiasing, and applications","volume":"38","author":"Zhang Xiaoying","year":"2019","unstructured":"Xiaoying Zhang , Hong Xie , Junzhou Zhao , and John CS Lui . 2019 . Understanding assimilation-contrast effects in online rating systems: modelling, debiasing, and applications . ACM Transactions on Information Systems (TOIS) 38 , 1 (2019), 1 -- 25 . Xiaoying Zhang, Hong Xie, Junzhou Zhao, and John CS Lui. 2019. Understanding assimilation-contrast effects in online rating systems: modelling, debiasing, and applications. ACM Transactions on Information Systems (TOIS) 38, 1 (2019), 1--25.","journal-title":"ACM Transactions on Information Systems (TOIS)"},{"key":"e_1_2_1_102_1","doi-asserted-by":"publisher","DOI":"10.1561\/1500000066"},{"key":"e_1_2_1_103_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.ijhm.2020.102658"},{"key":"e_1_2_1_104_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.ijhm.2020.102658"}],"container-title":["Proceedings of the ACM on Human-Computer Interaction"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3555597","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3555597","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T17:49:03Z","timestamp":1750182543000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3555597"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,11,7]]},"references-count":105,"journal-issue":{"issue":"CSCW2","published-print":{"date-parts":[[2022,11,7]]}},"alternative-id":["10.1145\/3555597"],"URL":"https:\/\/doi.org\/10.1145\/3555597","relation":{},"ISSN":["2573-0142"],"issn-type":[{"value":"2573-0142","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,11,7]]},"assertion":[{"value":"2022-11-11","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}