{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,14]],"date-time":"2026-04-14T00:52:04Z","timestamp":1776127924952,"version":"3.50.1"},"reference-count":105,"publisher":"Oxford University Press (OUP)","issue":"3","license":[{"start":{"date-parts":[[2024,3,21]],"date-time":"2024-03-21T00:00:00Z","timestamp":1710979200000},"content-version":"vor","delay-in-days":6,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2026,4,13]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>Labeling is critical in creating training datasets for supervised machine learning, and is a common form of crowd work heteromation. It typically requires manual labor, is badly compensated and not infrequently bores the workers involved. Although task variety is known to drive human autonomy and intrinsic motivation, there is little research in this regard in the labeling context. Against this backdrop, we manipulate the presentation sequence of a labeling task in an online experiment and use the theoretical lens of self-determination theory to explain psychological work outcomes and work performance. We rely on 176 crowd workers contributing with group comparisons between three presentation sequences (by label, by image, random) and a mediation path analysis along the phenomena studied. Surprising among our key findings is that the task variety when sorting by label is perceived higher than when sorting by image and the random group. Naturally, one would assume that the random group would be perceived as most varied. We choose a visual metaphor to explain this phenomenon, whereas paintings offer a structured presentation of coloured pixels, as opposed to random noise.<\/jats:p>","DOI":"10.1093\/iwc\/iwae008","type":"journal-article","created":{"date-parts":[[2024,2,22]],"date-time":"2024-02-22T13:59:36Z","timestamp":1708610376000},"page":"352-366","source":"Crossref","is-referenced-by-count":0,"title":["Paintings, Not Noise\u2014The Role of Presentation Sequence in Labeling"],"prefix":"10.1093","volume":"38","author":[{"given":"Merlin","family":"Knaeble","sequence":"first","affiliation":[{"name":"Human-Centered Systems Lab (h-lab) , Karlsruhe Institute of Technology (KIT), 76133 Karlsruhe,","place":["Germany"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Mario","family":"Nadj","sequence":"additional","affiliation":[{"name":"Business & Information Systems Engineering , TU Dortmund University, 44221 Dortmund,","place":["Germany"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Alexander","family":"Maedche","sequence":"additional","affiliation":[{"name":"Human-Centered Systems Lab (h-lab) , Karlsruhe Institute of Technology (KIT), 76133 Karlsruhe,","place":["Germany"]}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"286","published-online":{"date-parts":[[2024,3,15]]},"reference":[{"key":"2026041320045288000_ref1","first-page":"xviii, 317","volume-title":"Creativity in Context: Update to \u201cThe Social Psychology of Creativity\u201d","author":"Amabile","year":"1996"},{"key":"2026041320045288000_ref2","doi-asserted-by":"publisher","first-page":"105","DOI":"10.1609\/aimag.v35i4.2513","article-title":"Power to the people: the role of humans in interactive machine learning","volume":"35","author":"Amershi","year":"2014","journal-title":"AI Mag."},{"key":"2026041320045288000_ref3","first-page":"4011","article-title":"What drives the drivers? A qualitative perspective on what motivates the crowd delivery workforce","volume-title":"Proceedings of the 53th Annual Hawaii International Conference on System Sciences (HICSS)","author":"Asdecker","year":"2020"},{"key":"2026041320045288000_ref4","doi-asserted-by":"crossref","first-page":"223","DOI":"10.1016\/j.ijhcs.2009.12.003","article-title":"Better to be frustrated than bored: the incidence, persistence, and impact of learners\u2019 cognitive\u2013affective states during interactions with three different computer-based learning environments","volume":"68","author":"Baker","year":"2010","journal-title":"Int. J. Hum. Comput. Stud."},{"key":"2026041320045288000_ref5","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3491101.3503702","article-title":"Self-determination theory in HCI: shaping a research agenda","volume-title":"Extended Abstracts of the 2022 CHI Conference on Human Factors in Computing Systems","author":"Ballou","year":"2022"},{"key":"2026041320045288000_ref6","doi-asserted-by":"publisher","first-page":"428","DOI":"10.1016\/0196-8858(91)90029-I","article-title":"The perception of randomness","volume":"12","author":"Bar-Hillel","year":"1991","journal-title":"Adv. Appl. Math."},{"key":"2026041320045288000_ref7","doi-asserted-by":"publisher","first-page":"107122","DOI":"10.1016\/j.chb.2021.107122","article-title":"Understanding the impact of control levels over emotion-aware chatbots","volume":"129","author":"Benke","year":"2022","journal-title":"Comput. Hum. Behav."},{"key":"2026041320045288000_ref8","doi-asserted-by":"publisher","first-page":"343","DOI":"10.1016\/0049-089X(72)90082-8","article-title":"A lower-bound method for the dimension-free measurement of internal consistency","volume":"1","author":"Bentler","year":"1972","journal-title":"Soc. Sci. Res."},{"key":"2026041320045288000_ref9","doi-asserted-by":"publisher","first-page":"137","DOI":"10.1007\/s11336-008-9100-1","article-title":"Alpha, dimension-free, and model-based internal consistency reliability","volume":"74","author":"Bentler","year":"2008","journal-title":"Psychometrika"},{"key":"2026041320045288000_ref10","doi-asserted-by":"publisher","first-page":"298","DOI":"10.1109\/TVCG.2017.2744818","article-title":"Comparing visual-interactive labeling with active learning: an experimental study","volume":"24","author":"Bernard","year":"2018","journal-title":"IEEE Trans. Vis. Comput. Graph."},{"key":"2026041320045288000_ref11","doi-asserted-by":"publisher","first-page":"1189","DOI":"10.1007\/s00371-018-1500-3","article-title":"VIAL: a unified process for visual interactive labeling","volume":"34","author":"Bernard","year":"2018","journal-title":"Vis. Comput."},{"key":"2026041320045288000_ref12","doi-asserted-by":"publisher","first-page":"370","DOI":"10.2307\/2095172","article-title":"Issues in the comparative measurement of political democracy","volume":"45","author":"Bollen","year":"1980","journal-title":"Am. Sociol. Rev."},{"key":"2026041320045288000_ref13","doi-asserted-by":"publisher","first-page":"108","DOI":"10.1109\/TAMD.2010.2051030","article-title":"Designing interactions for robot active learners","volume":"2","author":"Cakmak","year":"2010","journal-title":"IEEE Trans. Auton. Ment. Dev."},{"key":"2026041320045288000_ref14","doi-asserted-by":"publisher","first-page":"399","DOI":"10.1509\/jmkr.38.4.399.18908","article-title":"What to say when: advertising appeals in evolving markets","volume":"38","author":"Chandy","year":"2001","journal-title":"Journal of Marketing Research (JMR)"},{"key":"2026041320045288000_ref15","doi-asserted-by":"publisher","first-page":"216","DOI":"10.1007\/s11031-014-9450-1","article-title":"Basic psychological need satisfaction, need frustration, and need strength across four cultures","volume":"39","author":"Chen","year":"2015","journal-title":"Motiv. Emot."},{"key":"2026041320045288000_ref16","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3185515","article-title":"Using machine learning to support qualitative coding in social science: shifting the focus to ambiguity","volume":"8","author":"Chen","year":"2018","journal-title":"ACM Trans. Interact. Intell. Syst."},{"key":"2026041320045288000_ref17","volume-title":"Statistical Power Analysis for the Behavioral Sciences","author":"Cohen","year":"1988"},{"key":"2026041320045288000_ref18","volume-title":"The Hot Hand: The Mystery and Science of Streaks","author":"Cohen","year":"2020"},{"key":"2026041320045288000_ref19","doi-asserted-by":"publisher","first-page":"14380","DOI":"10.1021\/acs.est.0c04393","article-title":"Association between aquatic micropollutant dissipation and river sediment bacterial communities","volume":"54","author":"Coll","year":"2020","journal-title":"Environ. Sci. Technol."},{"key":"2026041320045288000_ref20","doi-asserted-by":"publisher","first-page":"297","DOI":"10.1007\/BF02310555","article-title":"Coefficient alpha and the internal structure of tests","volume":"16","author":"Cronbach","year":"1951","journal-title":"Psychometrika"},{"key":"2026041320045288000_ref21","doi-asserted-by":"publisher","DOI":"10.24251\/HICSS.2021.019","volume-title":"Design Foundations for AI Assisted Decision Making: A Self Determination Theory Approach","author":"De Vreede","year":"2021"},{"key":"2026041320045288000_ref22","doi-asserted-by":"publisher","DOI":"10.1007\/978-1-4899-2271-7","volume-title":"Intrinsic Motivation and Self-Determination in Human Behavior","author":"Deci","year":"1985"},{"key":"2026041320045288000_ref23","doi-asserted-by":"publisher","first-page":"227","DOI":"10.1207\/S15327965PLI110401","article-title":"The \u201cwhat\u201d and \u201cwhy\u201d of goal pursuits: human needs and the self-determination of behavior","volume":"11","author":"Deci","year":"2000","journal-title":"Psychol. Inq."},{"key":"2026041320045288000_ref24","doi-asserted-by":"crossref","first-page":"182","DOI":"10.1037\/a0012801","article-title":"Self-determination theory: a macrotheory of human motivation, development, and health","volume":"49","author":"Deci","year":"2008","journal-title":"Can. Psychol."},{"key":"2026041320045288000_ref25","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1037\/0022-3514.40.1.1","article-title":"Characteristics of the rewarder and intrinsic motivation of the rewardee","volume":"40","author":"Deci","year":"1981","journal-title":"J. Pers. Soc. Psychol."},{"key":"2026041320045288000_ref26","doi-asserted-by":"publisher","first-page":"19","DOI":"10.1146\/annurev-orgpsych-032516-113108","article-title":"Self-determination theory in work organizations: the state of a science","volume":"4","author":"Deci","year":"2017","journal-title":"Annu. Rev. Organ. Psychol. Organ. Behav."},{"key":"2026041320045288000_ref27","doi-asserted-by":"publisher","first-page":"1361","DOI":"10.1287\/orsc.2016.1094","article-title":"Reducing organizational rule breaking through task variety: how task design supports deliberative thinking","volume":"27","author":"Derfler-Rozin","year":"2016","journal-title":"Organ. Sci."},{"key":"2026041320045288000_ref28","doi-asserted-by":"publisher","first-page":"3931","DOI":"10.1145\/2858036.2858395","article-title":"Contextual autonomy support in video game play: a grounded theory","volume-title":"Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems","author":"Deterding","year":"2016"},{"key":"2026041320045288000_ref29","doi-asserted-by":"publisher","first-page":"146","DOI":"10.1007\/s11031-012-9294-5","article-title":"The effect of variety expectations on interest, enjoyment, and locus of causality in exercise","volume":"37","author":"Dimmock","year":"2013","journal-title":"Motiv. Emot."},{"key":"2026041320045288000_ref30","article-title":"Nonparametric MANOVA in Mann\u2013Whitney effects","author":"Dobler","year":"2018"},{"key":"2026041320045288000_ref31","doi-asserted-by":"publisher","first-page":"997","DOI":"10.1007\/s10463-019-00717-","article-title":"Nonparametric MANOVA in meaningful effects","volume":"72","author":"Dobler","year":"2020","journal-title":"Ann. Inst. Stat. Math."},{"key":"2026041320045288000_ref32","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3185517","article-title":"A review of user interface design for interactive machine learning","volume":"8","author":"Dudley","year":"2018","journal-title":"ACM Trans. Interact. Intell. Syst."},{"key":"2026041320045288000_ref33","doi-asserted-by":"publisher","first-page":"66","DOI":"10.1080\/07421222.2019.1705506","article-title":"The nature of crowd work and its effects on individuals\u2019 work perception","volume":"37","author":"Durward","year":"2020","journal-title":"J. Manag. Inf. Syst."},{"key":"2026041320045288000_ref34","doi-asserted-by":"publisher","DOI":"10.5210\/fm.v19i6.5331","article-title":"Heteromation and its (dis)contents: the invisible division of labor between humans and machines","author":"Ekbia","year":"2014","journal-title":"First Monday"},{"key":"2026041320045288000_ref35","doi-asserted-by":"publisher","first-page":"175","DOI":"10.3758\/BF03193146","article-title":"$G\\ast $power 3: a flexible statistical power analysis program for the social, behavioral, and biomedical sciences","volume":"39","author":"Faul","year":"2007","journal-title":"Behav. Res. Methods"},{"key":"2026041320045288000_ref36","doi-asserted-by":"publisher","first-page":"1149","DOI":"10.3758\/BRM.41.4.1149","article-title":"Statistical power analyses using g$\\ast $power 3.1: tests for correlation and regression analyses","volume":"41","author":"Faul","year":"2009","journal-title":"Behav. Res. Methods"},{"key":"2026041320045288000_ref37","doi-asserted-by":"publisher","first-page":"121","DOI":"10.1016\/j.ijhcs.2009.10.002","article-title":"Beyond performance: feature awareness in personalized interfaces","volume":"68","author":"Findlater","year":"2010","journal-title":"Int. J. Hum. Comput. Stud."},{"key":"2026041320045288000_ref38","doi-asserted-by":"publisher","first-page":"916","DOI":"10.1037\/0022-3514.59.5.916","article-title":"Controlling teaching strategies: undermining children\u2019s self-determination and performance","volume":"59","author":"Flink","year":"1990","journal-title":"J. Pers. Soc. Psychol."},{"key":"2026041320045288000_ref39","doi-asserted-by":"publisher","first-page":"39","DOI":"10.2307\/3151312","article-title":"Evaluating structural equation models with unobservable variables and measurement error","volume":"18","author":"Fornell","year":"1981","journal-title":"J. Mark. Res."},{"key":"2026041320045288000_ref40","doi-asserted-by":"publisher","first-page":"3315","DOI":"10.1016\/j.jbusres.2016.02.013","article-title":"The effect of mobile technology usage on work engagement and emotional exhaustion in Japan","volume":"69","author":"Fujimoto","year":"2016","journal-title":"J. Bus. Res."},{"key":"2026041320045288000_ref41","doi-asserted-by":"publisher","first-page":"247","DOI":"10.1037\/cap0000324","article-title":"How algorithmic management influences worker motivation: a self-determination theory perspective","volume":"63","author":"Gagn\u00e9","year":"2022","journal-title":"Can. Psychol."},{"key":"2026041320045288000_ref42","doi-asserted-by":"publisher","first-page":"378","DOI":"10.1038\/s44159-022-00056-w","article-title":"Understanding and shaping the future of work with self-determination theory","volume":"1","author":"Gagn\u00e9","year":"2022","journal-title":"Nat. Rev. Psychol."},{"key":"2026041320045288000_ref43","doi-asserted-by":"crossref","first-page":"791","DOI":"10.1162\/003355300554917","article-title":"Pay enough or don\u2019t pay at all","volume":"115","author":"Gneezy","year":"2000","journal-title":"Q. J. Econ."},{"key":"2026041320045288000_ref44","doi-asserted-by":"publisher","first-page":"175","DOI":"10.1023\/A:1005614228250","article-title":"On the assessment of situational intrinsic and extrinsic motivation: the situational motivation scale (SIMS)","volume":"24","author":"Guay","year":"2000","journal-title":"Motiv. Emot."},{"key":"2026041320045288000_ref45","doi-asserted-by":"publisher","first-page":"250","DOI":"10.1016\/0030-5073(76)90016-7","article-title":"Motivation through the design of work: test of a theory","volume":"16","author":"Hackman","year":"1976","journal-title":"Organ. Behav. Hum. Perf."},{"key":"2026041320045288000_ref46","doi-asserted-by":"publisher","first-page":"454","DOI":"10.1037\/a0015241","article-title":"Perceptions of randomness: why three heads are better than four","volume":"116","author":"Hahn","year":"2009","journal-title":"Psychol. Rev."},{"key":"2026041320045288000_ref47","volume-title":"Multivariate Data Analysis","author":"Hair","year":"2019"},{"key":"2026041320045288000_ref48","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3173574.3174023","article-title":"A data-driven analysis of workers\u2019 earnings on amazon mechanical turk","volume-title":"Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems\u2014CHI \u201818","author":"Hara","year":"2018"},{"key":"2026041320045288000_ref49","doi-asserted-by":"publisher","first-page":"11","DOI":"10.1145\/1512714.1512717","article-title":"User experience (UX): Towards an experiential perspective on product quality","volume-title":"Proceedings of the 20th Conference on l\u2019Interaction Homme-Machine","author":"Hassenzahl","year":"2008"},{"key":"2026041320045288000_ref50","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3579456","article-title":"Aligning crowdworker perspectives and feedback outcomes in crowd-feedback system design","volume":"7","author":"Haug","year":"2023","journal-title":"Proc. ACM Hum.-Comput. Interact."},{"key":"2026041320045288000_ref51","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3415181","article-title":"CrowdCog: a cognitive skill based system for heterogeneous task assignment and recommendation in crowdsourcing","volume":"4","author":"Hettiachchi","year":"2020","journal-title":"Proc. ACM Hum.-Comput. Interact."},{"key":"2026041320045288000_ref52","doi-asserted-by":"publisher","first-page":"133","DOI":"10.1080\/1359432X.2016.1241769","article-title":"Boredom at work: towards a dynamic spillover model of need satisfaction, work motivation, and work-related boredom","volume":"26","author":"van Hooff","year":"2017","journal-title":"Eur. J. Work Organ. Psychol."},{"key":"2026041320045288000_ref53","article-title":"Every pixel has a random color","author":"Huiberts","year":"2019"},{"key":"2026041320045288000_ref54","doi-asserted-by":"publisher","first-page":"117","DOI":"10.1016\/0148-2963(94)00116-V","article-title":"Print advertising at the component level: a cross-cultural comparison of the United States and Japan","volume":"34","author":"Javalgi","year":"1995","journal-title":"J. Bus. Res."},{"key":"2026041320045288000_ref55","doi-asserted-by":"publisher","first-page":"3394","DOI":"10.1109\/EMBC.2018.8513059","article-title":"Rapid annotation of seizures and interictal-ictal continuum EEG patterns","volume-title":"40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC)","author":"Jing","year":"2018"},{"key":"2026041320045288000_ref56","doi-asserted-by":"publisher","first-page":"2","DOI":"10.30844\/wi2020a1-knaeble","article-title":"Oracle or teacher? A systematic overview of research on interactive labeling for machine learning","volume-title":"Proceedings of the 15th International Conference on Wirtschaftsinformatik (WI2020, Zentrale Tracks)","author":"Knaeble","year":"2020"},{"key":"2026041320045288000_ref57","article-title":"Tools of trade of the next blue-collar job? antecedents, design features, and outcomes of interactive labeling systems","volume-title":"31st European Conference on Information Systems (ECIS) Research Papers","author":"Knaeble","year":"2023"},{"key":"2026041320045288000_ref58","volume-title":"Punished by Rewards: The Trouble With Gold Stars, Incentive Plans, a\u2019s, Praise, and Other Bribes","author":"Kohn","year":"1993"},{"key":"2026041320045288000_ref59","doi-asserted-by":"publisher","first-page":"1089","DOI":"10.1145\/3322276.3322379","article-title":"Effect of interaction design on the human experience with interactive reinforcement learning","volume-title":"Proceedings of the 2019 on Designing Interactive Systems Conference","author":"Krening","year":"2019"},{"key":"2026041320045288000_ref60","doi-asserted-by":"publisher","first-page":"406","DOI":"10.1109\/IMCTL.2015.7359631","article-title":"Gamification design framework based on SDT for student motivation","volume-title":"2015 International Conference on Interactive Mobile Communication Technologies and Learning (IMCL)","author":"Lamprinou","year":"2015"},{"key":"2026041320045288000_ref61","volume-title":"Why Big Tech Pays Poor Kenyans to Teach Self-Driving Cars","author":"Lee","year":"2018"},{"key":"2026041320045288000_ref62","doi-asserted-by":"publisher","first-page":"197","DOI":"10.2753\/MIS0742-1222260108","article-title":"Leveraging crowdsourcing: activation-supporting components for IT-based ideas competition","volume":"26","author":"Leimeister","year":"2009","journal-title":"J. Manag. Inf. Syst."},{"key":"2026041320045288000_ref63","doi-asserted-by":"publisher","first-page":"132","DOI":"10.1145\/1142405.1142428","article-title":"Randomness as a resource for design","volume-title":"Proceedings of the 6th Conference on Designing Interactive Systems","author":"Leong","year":"2006"},{"key":"2026041320045288000_ref64","doi-asserted-by":"publisher","first-page":"322","DOI":"10.1080\/07421222.2015.1029402","article-title":"Employees\u2019 exploration of complex systems: an integrative view","volume":"32","author":"Liang","year":"2015","journal-title":"J. Manag. Inf. Syst."},{"key":"2026041320045288000_ref65","doi-asserted-by":"publisher","first-page":"168","DOI":"10.1016\/j.chb.2017.11.040","article-title":"How intrinsic motivation and extrinsic incentives affect task effort in crowdsourcing contests: a mediated moderation model","volume":"81","author":"Liang","year":"2018","journal-title":"Comput. Hum. Behav."},{"key":"2026041320045288000_ref66","doi-asserted-by":"publisher","first-page":"403","DOI":"10.1145\/2971648.2971709","article-title":"TaskMe: multi-task allocation in mobile crowd sensing","volume-title":"Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing","author":"Liu","year":"2016"},{"key":"2026041320045288000_ref67","doi-asserted-by":"publisher","first-page":"1154","DOI":"10.1080\/07421222.2018.1523587","article-title":"Active community participation and crowdworking turnover: a longitudinal model and empirical test of three mechanisms","volume":"35","author":"Ma","year":"2018","journal-title":"J. Manag. Inf. Syst."},{"key":"2026041320045288000_ref68","doi-asserted-by":"publisher","first-page":"227","DOI":"10.1177\/1523422318757210","article-title":"Self-determination theory can help you generate performance and well-being in the workplace: a review of the literature","volume":"20","author":"Manganelli","year":"2018","journal-title":"Adv. Dev. Hum. Resour."},{"key":"2026041320045288000_ref69","volume-title":"The German Ideology","author":"Marx","year":"1932"},{"key":"2026041320045288000_ref70","doi-asserted-by":"publisher","DOI":"10.4324\/9781410601087","volume-title":"Test Theory: A Unified Treatment","author":"McDonald","year":"1999"},{"key":"2026041320045288000_ref71","doi-asserted-by":"publisher","first-page":"387","DOI":"10.1016\/j.paid.2010.10.029","article-title":"Psychological needs, motivation, and well-being: a test of self-determination theory across multiple domains","volume":"50","author":"Milyavskaya","year":"2011","journal-title":"Personal. Individ. Differ."},{"key":"2026041320045288000_ref72","volume-title":"Compositie met groot rood vlak, geel, zwart, grijs en blauw","author":"Mondriaan","year":"1921"},{"key":"2026041320045288000_ref73","doi-asserted-by":"publisher","first-page":"1321","DOI":"10.1037\/0021-9010.91.6.1321","article-title":"The work design questionnaire (WDQ): developing and validating a comprehensive measure for assessing job design and the nature of work","volume":"91","author":"Morgeson","year":"2006","journal-title":"J. Appl. Psychol."},{"key":"2026041320045288000_ref74","doi-asserted-by":"publisher","first-page":"1124","DOI":"10.1016\/j.jbusres.2006.09.018","article-title":"Efficiency versus effectiveness in business networks","volume":"59","author":"Mouzas","year":"2006","journal-title":"J. Bus. Res."},{"key":"2026041320045288000_ref75","doi-asserted-by":"publisher","first-page":"131","DOI":"10.1007\/s13218-020-00634-1","article-title":"Power to the oracle? Design principles for interactive labeling systems in machine learning","volume":"34","author":"Nadj","year":"2020","journal-title":"KI - K\u00fcnstliche Intelligenz"},{"key":"2026041320045288000_ref76","doi-asserted-by":"publisher","first-page":"330","DOI":"10.1037\/0033-295X.109.2.330","article-title":"The production and perception of randomness","volume":"109","author":"Nickerson","year":"2002","journal-title":"Psychol. Rev."},{"key":"2026041320045288000_ref77","doi-asserted-by":"publisher","first-page":"194","DOI":"10.1177\/1477878509104324","article-title":"Motivation in physical education classes: a self-determination theory perspective","volume":"7","author":"Ntoumanis","year":"2009","journal-title":"Theory Res. Educ."},{"key":"2026041320045288000_ref78","doi-asserted-by":"publisher","first-page":"43","DOI":"10.1016\/j.chb.2015.08.049","article-title":"Digital media use in the 2 h before bedtime is associated with sleep variables in university students","volume":"55","author":"Orzech","year":"2016","journal-title":"Comput. Hum. Behav."},{"key":"2026041320045288000_ref79","doi-asserted-by":"publisher","first-page":"857","DOI":"10.1287\/mnsc.49.7.857.16382","article-title":"Conceptualizing and measuring variety in the execution of organizational work processes","volume":"49","author":"Pentland","year":"2003","journal-title":"Manag. Sci."},{"key":"2026041320045288000_ref80","doi-asserted-by":"crossref","first-page":"797","DOI":"10.3389\/fpsyg.2018.00797","article-title":"Designing for motivation, engagement and wellbeing in digital experience","volume":"9","author":"Peters","year":"2018","journal-title":"Front. Psychol."},{"key":"2026041320045288000_ref81","doi-asserted-by":"publisher","first-page":"82","DOI":"10.1080\/07421222.2021.1870385","article-title":"Mitigating the adverse effect of monetary incentives on voluntary contributions online","volume":"38","author":"Qiao","year":"2021","journal-title":"J. Manag. Inf. Syst."},{"key":"2026041320045288000_ref82","volume-title":"Is \u2018Data Labeling\u2019 the New Blue-Collar Job of the AI Era","author":"Reese","year":"2016"},{"key":"2026041320045288000_ref83","doi-asserted-by":"publisher","first-page":"153","DOI":"10.1177\/101269028602100207","article-title":"Sporticuous consumption: an analysis of social status symbolism in sport ads","volume":"21","author":"Renson","year":"1986","journal-title":"Int. Rev. Sociol. Sport"},{"key":"2026041320045288000_ref84","doi-asserted-by":"publisher","first-page":"1","DOI":"10.18637\/jss.v048.i02","article-title":"Lavaan: an r package for structural equation modeling","volume":"48","author":"Rosseel","year":"2012","journal-title":"J. Stat. Softw."},{"key":"2026041320045288000_ref85","doi-asserted-by":"publisher","first-page":"68","DOI":"10.1037\/0003-066X.55.1.68","article-title":"Self-determination theory and the facilitation of intrinsic motivation, social development, and well-being","volume":"55","author":"Ryan","year":"2000","journal-title":"Am. Psychol."},{"key":"2026041320045288000_ref86","doi-asserted-by":"publisher","first-page":"550","DOI":"10.1037\/0022-3514.50.3.550","article-title":"Origins and pawns in the classroom: self-report and projective assessment of individual differences in children\u2019s perceptions","volume":"50","author":"Ryan","year":"1986","journal-title":"J. Pers. Soc. Psychol."},{"key":"2026041320045288000_ref87","doi-asserted-by":"publisher","first-page":"3187","DOI":"10.1145\/3308558.3313716","article-title":"TurkScanner: predicting the hourly wage of microtasks","volume-title":"The World Wide Web Conference\u2014WWW \u201819","author":"Saito","year":"2019"},{"key":"2026041320045288000_ref88","doi-asserted-by":"publisher","first-page":"971","DOI":"10.1007\/s12671-014-0338-7","article-title":"Mindfulness, work climate, and psychological need satisfaction in employee well-being","volume":"6","author":"Schultz","year":"2015","journal-title":"Mindfulness"},{"key":"2026041320045288000_ref89","article-title":"Computer Sciences Technical Report No. 1648","volume-title":"Active Learning Literature Survey","author":"Settles","year":"2009"},{"key":"2026041320045288000_ref90","doi-asserted-by":"publisher","first-page":"379","DOI":"10.1002\/j.1538-7305.1948.tb01338.x","article-title":"A mathematical theory of communication","volume":"27","author":"Shannon","year":"1948","journal-title":"Bell Syst. Tech. J."},{"key":"2026041320045288000_ref91","doi-asserted-by":"publisher","first-page":"221","DOI":"10.1177\/0263276404046068","article-title":"Automotive emotions: feeling the car","volume":"21","author":"Sheller","year":"2004","journal-title":"Theory Cult. Soc."},{"key":"2026041320045288000_ref92","doi-asserted-by":"publisher","first-page":"514","DOI":"10.1145\/3397481.3450668","article-title":"Towards guidelines for designing human-in-the-loop machine training interfaces","volume-title":"26th International Conference on Intelligent User Interfaces","author":"van der Stappen","year":"2021"},{"key":"2026041320045288000_ref93","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1186\/1477-7525-7-15","article-title":"Internal construct validity of the Warwick\u2013Edinburgh mental well-being scale (WEMWBS): a rasch analysis using data from the scottish health education population survey","volume":"7","author":"Stewart-Brown","year":"2009","journal-title":"Health Qual. Life Outcomes"},{"key":"2026041320045288000_ref94","doi-asserted-by":"publisher","first-page":"1453","DOI":"10.1177\/0018720814553471","article-title":"On the application of motivation theory to human factors\/ergonomics: motivational design principles for human\u2013technology interaction","volume":"56","author":"Szalma","year":"2014","journal-title":"Hum. Factors"},{"key":"2026041320045288000_ref95","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1177\/0022242919841034","article-title":"What drives virality (sharing) of online digital content? The critical role of information, emotion, and brand prominence","volume":"83","author":"Tellis","year":"2019","journal-title":"J. Mark."},{"key":"2026041320045288000_ref96","doi-asserted-by":"publisher","first-page":"184","DOI":"10.1016\/j.chb.2017.08.038","article-title":"Validity and mechanical turk: an assessment of exclusion methods and interactive experiments","volume":"77","author":"Thomas","year":"2017","journal-title":"Comput. Hum. Behav."},{"key":"2026041320045288000_ref97","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1109\/CVPR.2007.383282","article-title":"A face annotation framework with partial clustering and interactive labeling","volume-title":"2007 IEEE Conference on Computer Vision and Pattern Recognition","author":"Tian","year":"2007"},{"key":"2026041320045288000_ref98","doi-asserted-by":"publisher","first-page":"211","DOI":"10.1002\/npr.4040040302","article-title":"Assessing performance and productivity in white-collar organizations","volume":"4","author":"Tuttle","year":"1985","journal-title":"National Productivity Review"},{"key":"2026041320045288000_ref99","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3313831.3376723","article-title":"Self-determination theory in HCI games research: current uses and open questions","volume-title":"Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems","author":"Tyack","year":"2020"},{"key":"2026041320045288000_ref100","doi-asserted-by":"publisher","first-page":"981","DOI":"10.1348\/096317909X481382","article-title":"Capturing autonomy, competence, and relatedness at work: construction and initial validation of the work-related basic need satisfaction scale","volume":"83","author":"Van den Broeck","year":"2010","journal-title":"J. Occup. Organ. Psychol."},{"key":"2026041320045288000_ref101","doi-asserted-by":"publisher","first-page":"102449","DOI":"10.1016\/j.ijhcs.2020.102449","article-title":"Apps that motivate: a taxonomy of app features based on self-determination theory","volume":"140","author":"Villalobos-Z\u00fa\u00f1iga","year":"2020","journal-title":"Int. J. Hum. Comput. Stud."},{"key":"2026041320045288000_ref102","doi-asserted-by":"crossref","first-page":"338","DOI":"10.1287\/mksc.2013.0837","article-title":"Just the faces: exploring the effects of facial features in print advertising","volume":"33","author":"Xiao","year":"2014","journal-title":"Mark. Sci."},{"key":"2026041320045288000_ref103","volume-title":"How Cheap Labor Drives China\u2019s A.I. Ambitions","author":"Yuan","year":"2018"},{"key":"2026041320045288000_ref104","doi-asserted-by":"publisher","first-page":"706","DOI":"10.1007\/978-3-540-88688-452","article-title":"Active image labeling and its application to facial action labeling","volume-title":"Computer Vision\u2014ECCV 2008","author":"Zhang","year":"2008"},{"key":"2026041320045288000_ref105","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3491102.3517612","article-title":"OneLabeler: a flexible system for building data labeling tools","volume-title":"Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems","author":"Zhang","year":"2022"}],"container-title":["Interacting with Computers"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/iwc\/article-pdf\/38\/3\/352\/57193390\/iwae008.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/iwc\/article-pdf\/38\/3\/352\/57193390\/iwae008.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,4,14]],"date-time":"2026-04-14T00:05:03Z","timestamp":1776125103000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/iwc\/article\/38\/3\/352\/7629774"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,3,15]]},"references-count":105,"journal-issue":{"issue":"3","published-online":{"date-parts":[[2024,3,15]]},"published-print":{"date-parts":[[2026,4,13]]}},"URL":"https:\/\/doi.org\/10.1093\/iwc\/iwae008","relation":{},"ISSN":["0953-5438","1873-7951"],"issn-type":[{"value":"0953-5438","type":"print"},{"value":"1873-7951","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2026,5]]},"published":{"date-parts":[[2024,3,15]]}}}