{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,13]],"date-time":"2026-04-13T17:41:25Z","timestamp":1776102085308,"version":"3.50.1"},"reference-count":57,"publisher":"Association for Computing Machinery (ACM)","issue":"CHI PLAY","license":[{"start":{"date-parts":[[2021,10,5]],"date-time":"2021-10-05T00:00:00Z","timestamp":1633392000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Proc. ACM Hum.-Comput. Interact."],"published-print":{"date-parts":[[2021,10,5]]},"abstract":"<jats:p>Measuring theoretical concepts, so-called constructs, is a central challenge of Player Experience research. Building on recent work in HCI and psychology, we conducted a systematic literature review to study the transparency of measurement reporting. We accessed the ACM Digital Library to analyze all 48 full papers published at CHI PLAY 2020, of those, 24 papers used self-report measurements and were included in the full review. We assessed specifically, whether researchers reported What, How and Why they measured. We found that researchers matched their measures to the construct under study and that administrative details, such as number of points on a Likert-type scale, were frequently reported. However, definitions of the constructs to be measured and justifications for selecting a particular scale were sparse. Lack of transparency in these areas threaten the validity of singular studies, but further compromise the building of theories and accumulation of research knowledge in meta-analytic work. This work is limited to only assessing the current transparency of measurement reporting at CHI PLAY 2020, however we argue this constitutes a fair foundation to assess potential pitfalls. To address these pitfalls, we propose a prescriptive model of a measurement selection process, which aids researchers to systematically define their constructs, specify operationalizations, and justify why these measures were chosen. Future research employing this model should contribute to more transparency in measurement reporting. The research was funded through internal resources. All materials are available on https:\/\/osf.io\/4xz2v\/.<\/jats:p>","DOI":"10.1145\/3474660","type":"journal-article","created":{"date-parts":[[2021,10,6]],"date-time":"2021-10-06T18:59:48Z","timestamp":1633546788000},"page":"1-21","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":19,"title":["Transparency in Measurement Reporting"],"prefix":"10.1145","volume":"5","author":[{"given":"Lena Fanya","family":"Aeschbach","sequence":"first","affiliation":[{"name":"Universit\u00e4t Basel, Basel, Switzerland"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Sebastian A.C.","family":"Perrig","sequence":"additional","affiliation":[{"name":"Universit\u00e4t Basel, Basel, Switzerland"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Lorena","family":"Weder","sequence":"additional","affiliation":[{"name":"Universit\u00e4t Basel, Basel, Switzerland"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Klaus","family":"Opwis","sequence":"additional","affiliation":[{"name":"Universit\u00e4t Basel, Basel, Switzerland"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Florian","family":"Br\u00fchlmann","sequence":"additional","affiliation":[{"name":"Universit\u00e4t Basel, Basel, Switzerland"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2021,10,6]]},"reference":[{"key":"e_1_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.ijhcs.2019.102370"},{"key":"e_1_2_1_2_1","unstructured":"CHI PLAY ACM. 2021. Contribution Types and Evaluation Criteria. https:\/\/chiplay.acm.org\/2021\/table-1\/"},{"key":"e_1_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1146\/annurev-orgpsych-041015-062305"},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1037\/amp0000191"},{"key":"e_1_2_1_5_1","volume-title":"American Psychological Association","author":"American Educational Research Association","unstructured":"American Educational Research Association, American Psychological Association, National Council on Measurement in Education, Joint Committee on Standards for Educational, and Psychological Testing (US). 1999. Standards for educational and psychological testing. Amer Educational Research Assn, Washington, DC, USA."},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1145\/1978942.1979336"},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1177\/1090198113483139"},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11336-006--1447--6"},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1145\/3410404.3414259"},{"key":"e_1_2_1_10_1","volume-title":"Mekler","author":"Br\u00fchlmann Florian","year":"2018","unstructured":"Florian Br\u00fchlmann and Elisa D. Mekler. 2018. Surveys in Games User Research. In Games User Research, Anders Drachen, Pejman Mirza-Babaei, and Lennart Nacke (Eds.). Oxford University Press, Oxford, Chapter 9, 141--162."},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1145\/2702613.2732831"},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1145\/2658537.2658708"},{"key":"e_1_2_1_13_1","volume-title":"Statistical and methodological myths and urban legends: Doctrine, verity and fable in the organizational and social sciences","author":"Chan David","unstructured":"David Chan. 2009. So why ask me? Are self-report data really that bad. In Statistical and methodological myths and urban legends: Doctrine, verity and fable in the organizational and social sciences, Charles E Lance and Robert J Vandenberg (Eds.). Taylor & Francis, Abingdon, UK, Chapter 13, 309--336."},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1145\/3170427.3185377"},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10869-010--9181--6"},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1177\/147078530805000106"},{"key":"e_1_2_1_17_1","volume-title":"Ryan","author":"Deci Edward L.","year":"2003","unstructured":"Edward L. Deci and Richard M. Ryan. 2003. Intrinsic motivation inventory. Self-determination theory 267 (2003)."},{"key":"e_1_2_1_18_1","volume-title":"Scale development: Theory and applications","author":"DeVellis Robert F.","unstructured":"Robert F. DeVellis. 2016. Scale development: Theory and applications. Vol. 26. Sage publications, Los Angeles, CA, USA."},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1145\/3170427.3188395"},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.7861\/clinmedicine.1--6--478"},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1017\/CBO9780511820724"},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1177\/2515245920952393"},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1145\/1357054.1357074"},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1177\/1745691620970585"},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pbio"},{"key":"e_1_2_1_26_1","volume-title":"Educational and psychological measurement and evaluation","author":"Hopkins Kenneth D","unstructured":"Kenneth D Hopkins. 1998. Educational and psychological measurement and evaluation. Pearson, London, UK."},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.ijhcs.2005.06.002"},{"key":"e_1_2_1_28_1","volume-title":"Yvonne De Kort, and Karolien Poels","author":"Jsselsteijn Wijnand","year":"2013","unstructured":"Wijnand IJsselsteijn, Yvonne De Kort, and Karolien Poels. 2013. The Game Experience Questionnaire. Eindhoven: Technische Universiteit Eindhoven."},{"key":"e_1_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.ijhcs.2018.05.003"},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1183\/09031936.00110209"},{"key":"e_1_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1145\/3242671.3242683"},{"key":"e_1_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.ijhcs.2013.09.006"},{"key":"e_1_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.1080\/10447318.2014.930311"},{"key":"e_1_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.3758\/bf03336715"},{"key":"e_1_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.1177\/0092070303031003011"},{"key":"e_1_2_1_36_1","unstructured":"Regan Mandryk. 2020. PACM Statement. https:\/\/chiplay.acm.org\/2021\/pacm\/"},{"key":"e_1_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.1080\/15366367.2017.1348108"},{"key":"e_1_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.1093\/iwc"},{"key":"e_1_2_1_39_1","doi-asserted-by":"publisher","DOI":"10.1186\/s13643-021-01626--4"},{"key":"e_1_2_1_40_1","volume-title":"Measurement, design, and analysis: An integrated approach. psychology press","author":"Pedhazur Elazar J","unstructured":"Elazar J Pedhazur and Liora Pedhazur Schmelkin. 2013. Measurement, design, and analysis: An integrated approach. psychology press, Hove, East Sussex, UK."},{"key":"e_1_2_1_41_1","doi-asserted-by":"publisher","DOI":"10.1145\/3173574.3174035"},{"key":"e_1_2_1_42_1","volume-title":"Yvonne De Kort, and Wijnand IJsselsteijn","author":"Poels Karolien","year":"2007","unstructured":"Karolien Poels, Yvonne De Kort, and Wijnand IJsselsteijn. 2007. D3.3 : Game Experience Questionnaire: development of a self-report measure to assess the psychological impact of digital games. Eindhoven: Technische Universiteit Eindhoven."},{"key":"e_1_2_1_43_1","doi-asserted-by":"publisher","DOI":"10.1145\/3410404.3414235"},{"key":"e_1_2_1_44_1","volume-title":"The measurement of implicit antidemocratic trends. The authoritarian personality","author":"Sanford R Nevitt","year":"1950","unstructured":"R Nevitt Sanford, Theodor W Adorno, Else Frenkel-Brunswik, and Daniel J Levinson. 1950. The measurement of implicit antidemocratic trends. The authoritarian personality (1950), 222--279."},{"key":"e_1_2_1_45_1","doi-asserted-by":"publisher","DOI":"10.3758\/s13428-020-01462--9"},{"key":"e_1_2_1_46_1","doi-asserted-by":"publisher","DOI":"10.1177\/0539018405058216"},{"key":"e_1_2_1_47_1","doi-asserted-by":"publisher","DOI":"10.1037\/a0034177"},{"key":"e_1_2_1_48_1","volume-title":"Lasorsa","author":"Shoemaker Pamela","year":"2004","unstructured":"Pamela Shoemaker, James W. Tankard, and Dominic L. Lasorsa. 2004. Theoretical concepts: The building blocks of theory. In How to Build Social Science Theories. SAGE Publications, Inc., Los Angeles, CA, 15--36. https:\/\/doi.org\/10. 4135\/9781412990110"},{"key":"e_1_2_1_49_1","first-page":"119","article-title":"Using Operational Definitions in Research: A Best- Practices Approach","volume":"37","author":"Slife Brent D.","year":"2016","unstructured":"Brent D. Slife, Casey D. Wright, and Stephen C. Yanchar. 2016. Using Operational Definitions in Research: A Best- Practices Approach. The Journal of Mind and Behavior 37, 2 (2016), 119--139. http:\/\/www.jstor.org\/stable\/44631540","journal-title":"The Journal of Mind and Behavior"},{"key":"e_1_2_1_50_1","doi-asserted-by":"publisher","DOI":"10.1145\/3410404.3414234"},{"key":"e_1_2_1_51_1","doi-asserted-by":"publisher","DOI":"10.1145\/3313831.3376723"},{"key":"e_1_2_1_52_1","doi-asserted-by":"publisher","DOI":"10.1145\/3411764.3445230"},{"key":"e_1_2_1_53_1","doi-asserted-by":"publisher","DOI":"10.1177\/0959354320945036"},{"key":"e_1_2_1_54_1","doi-asserted-by":"publisher","DOI":"10.1145\/3410404.3414229"},{"key":"e_1_2_1_55_1","doi-asserted-by":"publisher","DOI":"10.1145\/3313831.3376448"},{"key":"e_1_2_1_56_1","doi-asserted-by":"publisher","DOI":"10.1037\/0021--9010.79.3.323"},{"key":"e_1_2_1_57_1","doi-asserted-by":"publisher","DOI":"10.1145\/2907069"}],"container-title":["Proceedings of the ACM on Human-Computer Interaction"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3474660","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3474660","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T16:18:42Z","timestamp":1750177122000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3474660"}},"subtitle":["A Systematic Literature Review of CHI PLAY"],"short-title":[],"issued":{"date-parts":[[2021,10,5]]},"references-count":57,"journal-issue":{"issue":"CHI PLAY","published-print":{"date-parts":[[2021,10,5]]}},"alternative-id":["10.1145\/3474660"],"URL":"https:\/\/doi.org\/10.1145\/3474660","relation":{"has-preprint":[{"id-type":"doi","id":"10.31234\/osf.io\/69eqn","asserted-by":"object"}]},"ISSN":["2573-0142"],"issn-type":[{"value":"2573-0142","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,10,5]]},"assertion":[{"value":"2021-10-06","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}