{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,2]],"date-time":"2026-06-02T01:37:27Z","timestamp":1780364247313,"version":"3.54.1"},"reference-count":193,"publisher":"Springer Science and Business Media LLC","issue":"5","license":[{"start":{"date-parts":[[2024,8,10]],"date-time":"2024-08-10T00:00:00Z","timestamp":1723248000000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2024,8,10]],"date-time":"2024-08-10T00:00:00Z","timestamp":1723248000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"name":"Otto-von-Guericke University Magdeburg","award":["Innovation Fund"],"award-info":[{"award-number":["Innovation Fund"]}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Empir Software Eng"],"published-print":{"date-parts":[[2024,9]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:sec>\n                <jats:title>Context:<\/jats:title>\n                <jats:p>Empirical studies with human participants (e.g., controlled experiments) are established methods in Software Engineering (SE) research to understand developers\u2019 activities or the pros and cons of a technique, tool, or practice. Various guidelines and recommendations on designing and conducting different types of empirical studies in SE exist. However, the use of financial incentives (i.e., paying participants to compensate for their effort and improve the validity of a study) is rarely mentioned<\/jats:p>\n              <\/jats:sec><jats:sec>\n                <jats:title>Objective:<\/jats:title>\n                <jats:p>In this article, we analyze and discuss the use of financial incentives for human-oriented SE experimentation to derive corresponding guidelines and recommendations for researchers. Specifically, we propose how to extend the current state-of-the-art and provide a better understanding of when and how to incentivize.<\/jats:p>\n              <\/jats:sec><jats:sec>\n                <jats:title>Method:<\/jats:title>\n                <jats:p>We captured the state-of-the-art in SE by performing a Systematic Literature Review (SLR) involving 105 publications from six conferences and five journals published in 2020 and 2021. Then, we conducted an interdisciplinary analysis based on guidelines from experimental economics and behavioral psychology, two disciplines that research and use financial incentives.<\/jats:p>\n              <\/jats:sec><jats:sec>\n                <jats:title>Results:<\/jats:title>\n                <jats:p>Our results show that financial incentives are sparsely used in SE experimentation, mostly as completion fees. Especially performance-based and task-related financial incentives (i.e., payoff functions) are not used, even though we identified studies for which the validity may benefit from tailored payoff functions. To tackle this issue, we contribute an overview of how experiments in SE may benefit from financial incentivisation, a guideline for deciding on their use, and 11 recommendations on how to design them.<\/jats:p>\n              <\/jats:sec><jats:sec>\n                <jats:title>Conclusions:<\/jats:title>\n                <jats:p>We hope that our contributions get incorporated into standards (e.g., the ACM SIGSOFT Empirical Standards), helping researchers understand whether the use of financial incentives is useful for their experiments and how to define a suitable incentivisation strategy.<\/jats:p>\n              <\/jats:sec>","DOI":"10.1007\/s10664-024-10517-w","type":"journal-article","created":{"date-parts":[[2024,8,10]],"date-time":"2024-08-10T10:02:38Z","timestamp":1723284158000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":2,"title":["Guidelines for using financial incentives in software-engineering experimentation"],"prefix":"10.1007","volume":"29","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-0283-248X","authenticated-orcid":false,"given":"Jacob","family":"Kr\u00fcger","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"G\u00fcl","family":"\u00c7al\u0131kl\u0131","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Dmitri","family":"Bershadskyy","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Siegmar","family":"Otto","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sarah","family":"Zabel","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Robert","family":"Heyer","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2024,8,10]]},"reference":[{"key":"10517_CR1","doi-asserted-by":"crossref","unstructured":"Abdellatif A, Badran K, Shihab E (2020) MSRBot: Using bots to answer questions from software repositories. Empirical Software Engineering 25(3)","DOI":"10.1007\/s10664-019-09788-5"},{"key":"10517_CR2","doi-asserted-by":"publisher","first-page":"110912","DOI":"10.1016\/j.jss.2021.110912","volume":"175","author":"L Addazi","year":"2021","unstructured":"Addazi L, Ciccozzi F (2021) Blended graphical and textual modelling for UML profiles: a proof-of-concept implementation and experiment. J Syst Softw 175:110912. https:\/\/doi.org\/10.1016\/j.jss.2021.110912","journal-title":"J Syst Softw"},{"key":"10517_CR3","doi-asserted-by":"publisher","first-page":"110840","DOI":"10.1016\/j.jss.2020.110840","volume":"171","author":"E Aghayi","year":"2021","unstructured":"Aghayi E, LaToza TD, Surendra P, Abolghasemi S (2021) Crowdsourced behavior-driven development. J Syst Softw 171:110840. https:\/\/doi.org\/10.1016\/j.jss.2020.110840","journal-title":"Crowdsourced behavior-driven development. J Syst Softw"},{"key":"10517_CR4","doi-asserted-by":"crossref","unstructured":"Aguinis H, Villamor I, Ramani RS (2021) MTurk research: review and recommendations. Journal of Management, 47(4)","DOI":"10.1177\/0149206320969787"},{"key":"10517_CR5","doi-asserted-by":"publisher","first-page":"106483","DOI":"10.1016\/j.infsof.2020.106483","volume":"131","author":"M Ahrens","year":"2021","unstructured":"Ahrens M, Schneider K (2021) Improving requirements specification use by transferring attention with eye tracking data. Inf Softw Technol 131:106483. https:\/\/doi.org\/10.1016\/j.infsof.2020.106483","journal-title":"Inf Softw Technol"},{"key":"10517_CR6","doi-asserted-by":"publisher","first-page":"110945","DOI":"10.1016\/j.jss.2021.110945","volume":"176","author":"R Alanazi","year":"2021","unstructured":"Alanazi R, Gharibi G, Lee Y (2021) Facilitating program comprehension with call graph multilevel hierarchical abstractions. J Syst Softw 176:110945. https:\/\/doi.org\/10.1016\/j.jss.2021.110945","journal-title":"J Syst Softw"},{"key":"10517_CR7","doi-asserted-by":"publisher","unstructured":"Alhamed M, Storer T (2021) Playing planning poker in crowds: human computation of software effort estimates. In: International Conference on Software Engineering (ICSE), IEEE, pp 1\u201312. https:\/\/doi.org\/10.1109\/ICSE43902.2021.00014","DOI":"10.1109\/ICSE43902.2021.00014"},{"key":"10517_CR8","doi-asserted-by":"crossref","unstructured":"Allodi L, Cremonini M, Massacci F, Shim W (2020) Measuring the accuracy of software vulnerability assessments: experiments with students and professionals. Empirical Software Engineering 25(2)","DOI":"10.1007\/s10664-019-09797-4"},{"key":"10517_CR9","doi-asserted-by":"crossref","unstructured":"Am\u00e1lio N, Briand LC, Kelsen P (2020) An experimental scrutiny of visual design modelling: VCL up against UML+OCL. Empirical Software Engineering 25(2)","DOI":"10.1007\/s10664-019-09784-9"},{"key":"10517_CR10","doi-asserted-by":"publisher","first-page":"106412","DOI":"10.1016\/j.infsof.2020.106412","volume":"129","author":"A Ampatzoglou","year":"2021","unstructured":"Ampatzoglou A, Arvanitou E, Ampatzoglou A, Avgeriou P, Tsintzira A, Chatzigeorgiou A (2021) Architectural decision-making as a financial investment: an industrial case study. Inf Softw Technol 129:106412. https:\/\/doi.org\/10.1016\/j.infsof.2020.106412","journal-title":"Inf Softw Technol"},{"issue":"7748","key":"10517_CR11","doi-asserted-by":"publisher","first-page":"305","DOI":"10.1038\/d41586-019-00857-9","volume":"567","author":"V Amrhein","year":"2019","unstructured":"Amrhein V, Greenland S, McShane B (2019) Retire statistical significance: scientists rise up against statistical significance. Nature 567(7748):305\u2013307. https:\/\/doi.org\/10.1038\/d41586-019-00857-9","journal-title":"Nature"},{"key":"10517_CR12","doi-asserted-by":"publisher","unstructured":"Azizi B, Zamani B, Rahimi SK (2020) SEET: symbolic execution of ETL transformations. J Syst Softw 168. https:\/\/doi.org\/10.1016\/j.jss.2020.110675","DOI":"10.1016\/j.jss.2020.110675"},{"key":"10517_CR13","doi-asserted-by":"publisher","unstructured":"Bai GR, Kayani J, Stolee KT (2020) How graduate computing students search when using an unfamiliar programming language. In: International Conference on Program Comprehension (ICPC), ACM, pp 160\u2013171. https:\/\/doi.org\/10.1145\/3387904.3389274","DOI":"10.1145\/3387904.3389274"},{"issue":"7593","key":"10517_CR14","doi-asserted-by":"publisher","first-page":"151","DOI":"10.1038\/nature.2016.19503","volume":"531","author":"M Baker","year":"2016","unstructured":"Baker M (2016) Statisticians issue warning over misuse of p values. Nature 531(7593):151\u2013151. https:\/\/doi.org\/10.1038\/nature.2016.19503","journal-title":"Nature"},{"key":"10517_CR15","doi-asserted-by":"publisher","first-page":"110937","DOI":"10.1016\/j.jss.2021.110937","volume":"176","author":"MT Baldassarre","year":"2021","unstructured":"Baldassarre MT, Caivano D, Fucci D, Juristo N, Romano S, Scanniello G, Turhan B (2021) Studying test-driven development and its retainment over a six-month time span. J Syst Softw 176:110937. https:\/\/doi.org\/10.1016\/j.jss.2021.110937","journal-title":"J Syst Softw"},{"issue":"3","key":"10517_CR16","doi-asserted-by":"publisher","first-page":"418","DOI":"10.1007\/s10683-011-9306-4","volume":"15","author":"G Baltussen","year":"2012","unstructured":"Baltussen G, Post GT, van den Assem MJ, Wakker PP (2012) Random incentive systems in a dynamic choice experiment. Exp Econ 15(3):418\u2013443","journal-title":"Exp Econ"},{"key":"10517_CR17","doi-asserted-by":"publisher","unstructured":"Bao L, Xing Z, Xia X, Lo D, Wu M, Yang X (2020) psc2code: denoising code extraction from programming screencasts. ACM Transactions on Software Engineering and Methodology, 29(3):21:1\u201321:38. https:\/\/doi.org\/10.1145\/3392093","DOI":"10.1145\/3392093"},{"key":"10517_CR18","doi-asserted-by":"publisher","first-page":"221","DOI":"10.1016\/j.ssresearch.2019.02.008","volume":"81","author":"R Becker","year":"2019","unstructured":"Becker R, M\u00f6ser S, Glauser D (2019) Cash vs. vouchers vs. gifts in web surveys of a mature panel study-main effects in a long-term incentives experiment across three panel waves. Soc Sci Res 81:221\u2013234","journal-title":"Soc Sci Res"},{"key":"10517_CR19","doi-asserted-by":"publisher","unstructured":"Behroozi M, Shirolkar S, Barik T, Parnin C (2020) Does stress impact technical interview performance? In: Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC\/FSE), ACM, pp 481\u2013492. https:\/\/doi.org\/10.1145\/3368089.3409712","DOI":"10.1145\/3368089.3409712"},{"key":"10517_CR20","doi-asserted-by":"publisher","unstructured":"Beschastnikh I, Liu P, Xing A, Wang P, Brun Y, Ernst MD (2020) Visualizing distributed system executions. ACM Transactions on Software Engineering and Methodology, 29(2):9:1\u20139:38. https:\/\/doi.org\/10.1145\/3375633","DOI":"10.1145\/3375633"},{"key":"10517_CR21","doi-asserted-by":"publisher","first-page":"110985","DOI":"10.1016\/j.jss.2021.110985","volume":"179","author":"JZ Blanco","year":"2021","unstructured":"Blanco JZ, Lucr\u00e9dio D (2021) A holistic approach for cross-platform software development. J Syst Softw 179:110985. https:\/\/doi.org\/10.1016\/j.jss.2021.110985","journal-title":"J Syst Softw"},{"key":"10517_CR22","doi-asserted-by":"publisher","unstructured":"Braz L, Fregnan E, \u00c7alikli G, Bacchelli A (2021) Why don\u2019t developers detect improper input validation? \u2019; DROP TABLE Papers; \u2013. In: International Conference on Software Engineering (ICSE), IEEE, pp 499\u2013511. https:\/\/doi.org\/10.1109\/ICSE43902.2021.00054","DOI":"10.1109\/ICSE43902.2021.00054"},{"issue":"3","key":"10517_CR23","doi-asserted-by":"publisher","first-page":"369","DOI":"10.2501\/IJMR-53-3-369-390","volume":"53","author":"E Br\u00fcggen","year":"2011","unstructured":"Br\u00fcggen E, Wetzels M, De Ruyter K, Schillewaert N (2011) Individual differences in motivation to participate in online panels: the effect on reponse rate and reponse quality perceptions. Int J Mark Res 53(3):369\u2013390","journal-title":"Int J Mark Res"},{"issue":"1","key":"10517_CR24","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1086\/261439","volume":"95","author":"C Bull","year":"1987","unstructured":"Bull C, Schotter A, Weigelt K (1987) Tournaments and piece rates: an experimental study. J Polit Econ 95(1):1\u201333","journal-title":"J Polit Econ"},{"key":"10517_CR25","doi-asserted-by":"crossref","unstructured":"Burtch G, Hong Y, Bapna R, Griskevicius V (2018) Stimulating online reviews by combining financial incentives and social norms. Management Science, 64(5)","DOI":"10.1287\/mnsc.2016.2715"},{"issue":"1","key":"10517_CR26","doi-asserted-by":"publisher","first-page":"7","DOI":"10.1023\/A:1007850605129","volume":"19","author":"CF Camerer","year":"1999","unstructured":"Camerer CF, Hogarth RM (1999) The effects of financial incentives in experiments: a review and capital-labor-production framework. J Risk Uncertain 19(1):7\u201342","journal-title":"J Risk Uncertain"},{"issue":"1","key":"10517_CR27","doi-asserted-by":"publisher","first-page":"46","DOI":"10.1016\/j.tics.2016.11.001","volume":"21","author":"CF Camerer","year":"2017","unstructured":"Camerer CF, Mobbs D (2017) Differences in behavior and brain activity during hypothetical and real choices. Trends Cogn Sci 21(1):46\u201356","journal-title":"Trends Cogn Sci"},{"key":"10517_CR28","doi-asserted-by":"crossref","unstructured":"Carpenter J, Huet-Vaughn E (2019) Real-effort tasks. Handbook of Research Methods and Applications in Experimental Economics","DOI":"10.4337\/9781788110563.00030"},{"key":"10517_CR29","doi-asserted-by":"publisher","first-page":"35","DOI":"10.1007\/s10664-009-9109-9","volume":"15","author":"JC Carver","year":"2010","unstructured":"Carver JC, Jaccheri L, Morasca S, Shull F (2010) A checklist for integrating student empirical studies with research and teaching goals. Empir Softw Eng 15:35\u201359","journal-title":"Empir Softw Eng"},{"issue":"9\u201310","key":"10517_CR30","doi-asserted-by":"publisher","first-page":"604","DOI":"10.1016\/j.jpubeco.2010.05.006","volume":"94","author":"TN Cason","year":"2010","unstructured":"Cason TN, Masters WA, Sheremeta RM (2010) Entry Into Winner-Take-All and Proportional-Prize Contests: An Experimental Study. J Public Econ 94(9\u201310):604\u2013611","journal-title":"J Public Econ"},{"key":"10517_CR31","doi-asserted-by":"publisher","unstructured":"Cates R, Yunik N, Feitelson DG (2021) Does code structure affect comprehension? On using and naming intermediate variables. In: International Conference on Program Comprehension (ICPC), IEEE, pp 118\u2013126. https:\/\/doi.org\/10.1109\/ICPC52881.2021.00020","DOI":"10.1109\/ICPC52881.2021.00020"},{"key":"10517_CR32","doi-asserted-by":"publisher","unstructured":"Caulo M, Francese R, Scanniello G, Tortora G (2021) Relationships between personality traits and productivity in a multi-platform development context. In: International Conference on Evaluation and Assessment in Software Engineering (EASE), ACM, pp 70\u201379. https:\/\/doi.org\/10.1145\/3463274.3463327","DOI":"10.1145\/3463274.3463327"},{"issue":"4","key":"10517_CR33","doi-asserted-by":"publisher","first-page":"980","DOI":"10.1037\/a0035661","volume":"140","author":"CP Cerasoli","year":"2014","unstructured":"Cerasoli CP, Nicklin JM, Ford MT (2014) Intrinsic motivation and extrinsic incentives jointly predict performance: a 40-year meta-analysis. Psychol Bull 140(4):980\u20131008. https:\/\/doi.org\/10.1037\/a0035661","journal-title":"Psychol Bull"},{"key":"10517_CR34","doi-asserted-by":"publisher","unstructured":"Chattopadhyay S, Nelson N, Au A, Morales N, Sanchez C, Pandita R, Sarma A (2020) A tale from the trenches: cognitive biases and software development. In: International Conference on Software Engineering (ICSE), ACM, pp 654\u2013665. https:\/\/doi.org\/10.1145\/3377811.3380330","DOI":"10.1145\/3377811.3380330"},{"key":"10517_CR35","doi-asserted-by":"publisher","unstructured":"Cornejo O, Briola D, Micucci D, Mariani L (2020) In-the-field monitoring of functional calls: is it feasible? J Syst Softw 163. https:\/\/doi.org\/10.1016\/j.jss.2020.110523","DOI":"10.1016\/j.jss.2020.110523"},{"key":"10517_CR36","doi-asserted-by":"publisher","unstructured":"Corradini F, Morichetta A, Polini A, Re B, Rossi L, Tiezzi F (2020) Correctness checking for BPMN collaborations with sub-processes. J Syst Softw 166. https:\/\/doi.org\/10.1016\/j.jss.2020.110594","DOI":"10.1016\/j.jss.2020.110594"},{"issue":"5","key":"10517_CR37","doi-asserted-by":"publisher","first-page":"92","DOI":"10.1007\/s10664-021-10002-8","volume":"26","author":"JAS da Costa","year":"2021","unstructured":"da Costa JAS, Gheyi R, Ribeiro M, Apel S, Alves V, Fonseca B, Medeiros F, Garcia A (2021) Evaluating refactorings for disciplining #Ifdef annotations: an eye tracking study with novices. Empir Softw Eng 26(5):92. https:\/\/doi.org\/10.1007\/s10664-021-10002-8","journal-title":"Empir Softw Eng"},{"issue":"2","key":"10517_CR38","doi-asserted-by":"publisher","first-page":"115","DOI":"10.1023\/A:1026435508449","volume":"1","author":"RP Cubitt","year":"1998","unstructured":"Cubitt RP, Starmer C, Sugden R (1998) On the validity of the random lottery incentive system. Exp Econ 1(2):115\u2013131","journal-title":"Exp Econ"},{"issue":"1","key":"10517_CR39","doi-asserted-by":"publisher","first-page":"100","DOI":"10.1109\/TSE.2018.2859926","volume":"46","author":"C Czepa","year":"2020","unstructured":"Czepa C, Zdun U (2020) On the understandability of temporal properties formalized in linear temporal logic, property specification patterns and event processing language. IEEE Trans Software Eng 46(1):100\u2013112. https:\/\/doi.org\/10.1109\/TSE.2018.2859926","journal-title":"IEEE Trans Software Eng"},{"key":"10517_CR40","doi-asserted-by":"publisher","first-page":"106484","DOI":"10.1016\/j.infsof.2020.106484","volume":"131","author":"F Dalpiaz","year":"2021","unstructured":"Dalpiaz F, Gieske P, Sturm A (2021) On deriving conceptual models drom user requirements: an empirical study. Inf Softw Technol 131:106484. https:\/\/doi.org\/10.1016\/j.infsof.2020.106484","journal-title":"Inf Softw Technol"},{"key":"10517_CR41","doi-asserted-by":"publisher","unstructured":"Danilova A, Naiakshina A, Horstmann S, Smith M (2021) Do you really code? designing and evaluating screening questions for online surveys with programmers. In: International Conference on Software Engineering (ICSE), IEEE, pp 537\u2013548. https:\/\/doi.org\/10.1109\/ICSE43902.2021.00057","DOI":"10.1109\/ICSE43902.2021.00057"},{"issue":"11","key":"10517_CR42","doi-asserted-by":"publisher","first-page":"1210","DOI":"10.1016\/j.jclinepi.2014.08.001","volume":"67","author":"MC David","year":"2014","unstructured":"David MC, Ware RS (2014) Meta-analysis of randomized controlled trials supports the use of incentives for inducing response to electronic health surveys. J Clin Epidemiol 67(11):1210\u20131221","journal-title":"J Clin Epidemiol"},{"issue":"1","key":"10517_CR43","doi-asserted-by":"publisher","first-page":"105","DOI":"10.1037\/h0030644","volume":"18","author":"EL Deci","year":"1971","unstructured":"Deci EL (1971) Effects of externally mediated rewards on intrinsic motivation. J Pers Soc Psychol 18(1):105","journal-title":"J Pers Soc Psychol"},{"issue":"2","key":"10517_CR44","doi-asserted-by":"publisher","first-page":"1029","DOI":"10.1093\/restud\/rdx033","volume":"85","author":"S DellaVigna","year":"2018","unstructured":"DellaVigna S, Pope D (2018) What motivates effort? Evidence and expert forecasts. Rev Econ Stud 85(2):1029\u20131069","journal-title":"Rev Econ Stud"},{"key":"10517_CR45","doi-asserted-by":"publisher","unstructured":"Dias M, Orellana D, Vidal SA, Merino L, Bergel A (2020) Evaluating a Visual Approach for Understanding JavaScript Source Code. In: International Conference on Program Comprehension (ICPC), ACM, pp 128\u2013138. https:\/\/doi.org\/10.1145\/3387904.3389275","DOI":"10.1145\/3387904.3389275"},{"issue":"2","key":"10517_CR46","doi-asserted-by":"publisher","first-page":"187","DOI":"10.1016\/s0014-2921(00)00056-8","volume":"45","author":"F van Dijk","year":"2001","unstructured":"van Dijk F, Sonnemans J, van Winden F (2001) Incentive systems in a real effort experiment. Eur Econ Rev 45(2):187\u2013214. https:\/\/doi.org\/10.1016\/s0014-2921(00)00056-8","journal-title":"Eur Econ Rev"},{"issue":"7","key":"10517_CR47","doi-asserted-by":"publisher","first-page":"697","DOI":"10.1109\/TSE.2018.2868349","volume":"46","author":"LNQ Do","year":"2020","unstructured":"Do LNQ, Kr\u00fcger S, Hill P, Ali K, Bodden E (2020) Debugging static analysis. IEEE Trans Software Eng 46(7):697\u2013709. https:\/\/doi.org\/10.1109\/TSE.2018.2868349","journal-title":"IEEE Trans Software Eng"},{"key":"10517_CR48","doi-asserted-by":"publisher","first-page":"106444","DOI":"10.1016\/j.infsof.2020.106444","volume":"130","author":"J Echeverr\u00eda","year":"2021","unstructured":"Echeverr\u00eda J, P\u00e9rez F, Panach JI, Cetina C (2021) An empirical study of performance using clone & own and software product lines in an industrial context. Inf Softw Technol 130:106444. https:\/\/doi.org\/10.1016\/j.infsof.2020.106444","journal-title":"Inf Softw Technol"},{"issue":"11","key":"10517_CR49","doi-asserted-by":"publisher","first-page":"987","DOI":"10.1136\/jech.2005.034397","volume":"59","author":"P Edwards","year":"2005","unstructured":"Edwards P, Cooper R, Roberts I, Frost C (2005) Meta-analysis of randomised trials of monetary incentives and response to mailed questionnaires. Journal of Epidemiology & Community Health 59(11):987\u2013999","journal-title":"Journal of Epidemiology & Community Health"},{"key":"10517_CR50","doi-asserted-by":"publisher","unstructured":"Endres M, Fansher M, Shah P, Weimer W (2021a) To read or to rotate? Comparing the effects of technical reading training and spatial skills training on novice programming ability. In: Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC\/FSE), ACM, pp 754\u2013766. https:\/\/doi.org\/10.1145\/3468264.3468583","DOI":"10.1145\/3468264.3468583"},{"key":"10517_CR51","doi-asserted-by":"publisher","unstructured":"Endres M, Karas Z, Hu X, Kovelman I, Weimer W (2021b) Relating reading, visualization, and coding for new programmers: a neuroimaging study. In: International Conference on Software Engineering (ICSE), IEEE, pp 600\u2013612. https:\/\/doi.org\/10.1109\/ICSE43902.2021.00062","DOI":"10.1109\/ICSE43902.2021.00062"},{"key":"10517_CR52","doi-asserted-by":"publisher","first-page":"528","DOI":"10.1016\/j.euroecorev.2017.10.021","volume":"101","author":"N Erkal","year":"2018","unstructured":"Erkal N, Gangadharan L, Koh BH (2018) Monetary and non-monetary incentives in real-effort tournaments. Eur Econ Rev 101:528\u2013545. https:\/\/doi.org\/10.1016\/j.euroecorev.2017.10.021","journal-title":"Eur Econ Rev"},{"key":"10517_CR53","unstructured":"Esteves-Sorenson C, Broce R (2020) Do monetary incentives undermine performance on intrinsically enjoyable tasks? A field test. Review of Economics and Statistics pp 1\u201346"},{"issue":"3","key":"10517_CR54","doi-asserted-by":"publisher","first-page":"2140","DOI":"10.1007\/s10664-019-09751-4","volume":"25","author":"S Fakhoury","year":"2020","unstructured":"Fakhoury S, Roy D, Ma Y, Arnaoudova V, Adesope OO (2020) Measuring the impact of lexical and structural inconsistencies on developers\u2019 cognitive load during bug localization. Empir Softw Eng 25(3):2140\u20132178. https:\/\/doi.org\/10.1007\/s10664-019-09751-4","journal-title":"Empir Softw Eng"},{"key":"10517_CR55","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-32489-6","author":"M Felderer","year":"2020","unstructured":"Felderer M, Travassos GH (2020) Contemporary empirical methods in software engineering. Springer. https:\/\/doi.org\/10.1007\/978-3-030-32489-6","journal-title":"Springer"},{"issue":"2","key":"10517_CR56","doi-asserted-by":"publisher","first-page":"371","DOI":"10.1111\/j.1467-6419.2010.00676.x","volume":"25","author":"N Feltovich","year":"2011","unstructured":"Feltovich N (2011) What\u2019s to know about laboratory experimentation in economics? Journal of Economic Surveys 25(2):371\u2013379","journal-title":"Journal of Economic Surveys"},{"key":"10517_CR57","doi-asserted-by":"publisher","unstructured":"Fiore AT, Cheshire C, Taylor L, Mendelsohn GA (2014) Incentives to participate in online research: an experimental examination of \u201csurprise\" incentives. In: International Conference on Human Factors in Computing Systems (CHI), ACM, pp 3433\u20133442. https:\/\/doi.org\/10.1145\/2556288.2557418","DOI":"10.1145\/2556288.2557418"},{"key":"10517_CR58","doi-asserted-by":"publisher","unstructured":"Foundjem A, Eghan EE, Adams B (2021) Onboarding vs. diversity, productivity, and quality - empirical study of the Openstack ecosystem. In: International Conference on Software Engineering (ICSE), IEEE, pp 1033\u20131045. https:\/\/doi.org\/10.1109\/ICSE43902.2021.00097","DOI":"10.1109\/ICSE43902.2021.00097"},{"key":"10517_CR59","unstructured":"Frey BS (1997) Not Just for the Money. Edward Elgar Publishing"},{"issue":"1","key":"10517_CR60","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1109\/TSE.2018.2834900","volume":"46","author":"D Fucci","year":"2020","unstructured":"Fucci D, Scanniello G, Romano S, Juristo N (2020) Need for sleep: the impact of a night of sleep deprivation on novice developers\u2019 performance. IEEE Trans Software Eng 46(1):1\u201319. https:\/\/doi.org\/10.1109\/TSE.2018.2834900","journal-title":"IEEE Trans Software Eng"},{"key":"10517_CR61","doi-asserted-by":"publisher","unstructured":"Gil M, Albert M, Fons J, Pelechano V (2020) Engineering human-in-the-loop interactions in cyber-physical systems. Inf Softw Technol 126. https:\/\/doi.org\/10.1016\/j.infsof.2020.106349","DOI":"10.1016\/j.infsof.2020.106349"},{"key":"10517_CR62","doi-asserted-by":"publisher","unstructured":"Girardi D, Novielli N, Fucci D, Lanubile F (2020) Recognizing developers\u2019 emotions while programming. In: International Conference on Software Engineering (ICSE), ACM, pp 666\u2013677. https:\/\/doi.org\/10.1145\/3377811.3380374","DOI":"10.1145\/3377811.3380374"},{"key":"10517_CR63","doi-asserted-by":"publisher","unstructured":"Glasgow MJ, Murphy MS (1992) An experiment with financial incentives for a small software development team. In: Washington Ada Symposium on Ada: Empowering Software Users and Developers (WADAS), ACM, pp 86\u201392. https:\/\/doi.org\/10.1145\/257683.257713","DOI":"10.1145\/257683.257713"},{"issue":"3","key":"10517_CR64","doi-asserted-by":"publisher","first-page":"791","DOI":"10.1162\/003355300554917","volume":"115","author":"U Gneezy","year":"2000","unstructured":"Gneezy U, Rustichini A (2000) Pay enough or don\u2019t pay at all. Q J Econ 115(3):791\u2013810","journal-title":"Q J Econ"},{"key":"10517_CR65","doi-asserted-by":"publisher","unstructured":"Gopstein D, Fayard A, Apel S, Cappos J (2020) Thinking aloud about confusing code: a qualitative investigation of program comprehension and atoms of confusion. In: Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC\/FSE), ACM, pp 605\u2013616. https:\/\/doi.org\/10.1145\/3368089.3409714","DOI":"10.1145\/3368089.3409714"},{"key":"10517_CR66","doi-asserted-by":"publisher","unstructured":"Goumopoulos C, Mavrommati I (2020) A framework for pervasive computing applications based on smart objects and end user development. J Syst Softw 162. https:\/\/doi.org\/10.1016\/j.jss.2019.110496","DOI":"10.1016\/j.jss.2019.110496"},{"issue":"6","key":"10517_CR67","doi-asserted-by":"publisher","first-page":"5416","DOI":"10.1007\/s10664-020-09883-y","volume":"25","author":"C Gralha","year":"2020","unstructured":"Gralha C, Goul\u00e3o M, Ara\u00fajo J (2020) Are there gender differences when interacting with social goal models? Empir Softw Eng 25(6):5416\u20135453. https:\/\/doi.org\/10.1007\/s10664-020-09883-y","journal-title":"Empir Softw Eng"},{"key":"10517_CR68","doi-asserted-by":"publisher","unstructured":"Grossklags J (2007) Experimental economics and experimental computer science: a survey. In: Workshop on Experimental Computer Science (ExpCS), ACM, https:\/\/doi.org\/10.1145\/1281700.1281711","DOI":"10.1145\/1281700.1281711"},{"key":"10517_CR69","doi-asserted-by":"publisher","unstructured":"Guerriero M, Tamburri DA, Nitto ED (2021) StreamGen: model-driven development of distributed streaming applications. ACM Transactions on Software Engineering and Methodology 30(1):1:1\u20131:30. https:\/\/doi.org\/10.1145\/3408895","DOI":"10.1145\/3408895"},{"issue":"4","key":"10517_CR70","doi-asserted-by":"publisher","first-page":"380","DOI":"10.1016\/j.jretai.2018.07.002","volume":"94","author":"K Gunasti","year":"2018","unstructured":"Gunasti K, Baskin E (2018) Is a $${\\$200}$$ nordstrom gift card worth more or less than a $${\\$200}$$ gap gift card? the asymmetric valuations of luxury gift cards. J Retail 94(4):380\u2013392","journal-title":"J Retail"},{"key":"10517_CR71","doi-asserted-by":"publisher","unstructured":"Hallett J, Patnaik N, Shreeve B, Rashid A (2021) \u201cDo this! do that!, and nothing will happen\" do specifications lead to securely stored passwords? In: IEEE, IEEE, pp 486\u2013498. https:\/\/doi.org\/10.1109\/ICSE43902.2021.00053","DOI":"10.1109\/ICSE43902.2021.00053"},{"issue":"5","key":"10517_CR72","first-page":"1426","volume":"82","author":"GW Harrison","year":"1992","unstructured":"Harrison GW (1992) Theory and misbehavior of first-price auctions: Reply. Am Econ Rev 82(5):1426\u20131443","journal-title":"Am Econ Rev"},{"issue":"4","key":"10517_CR73","doi-asserted-by":"publisher","first-page":"1009","DOI":"10.1257\/0022051043004577","volume":"42","author":"GW Harrison","year":"2004","unstructured":"Harrison GW, List JA (2004) Field experiments. Journal of Economic literature 42(4):1009\u20131055","journal-title":"Field experiments. Journal of Economic literature"},{"issue":"3","key":"10517_CR74","doi-asserted-by":"publisher","first-page":"498","DOI":"10.1016\/j.jebo.2008.02.011","volume":"70","author":"GW Harrison","year":"2009","unstructured":"Harrison GW, Lau MI, Rutstr\u00f6m EE (2009) Risk attitudes, randomization to treatment, and self-selection into experiments. Journal of Economic Behavior & Organization 70(3):498\u2013507","journal-title":"Journal of Economic Behavior & Organization"},{"issue":"3","key":"10517_CR75","doi-asserted-by":"publisher","first-page":"383","DOI":"10.1017\/S0140525X01004149","volume":"24","author":"R Hertwig","year":"2001","unstructured":"Hertwig R, Ortmann A (2001) Experimental practices in economics: a methodological challenge for psychologists? Behavioral and Brain Sciences 24(3):383\u2013403","journal-title":"Behavioral and Brain Sciences"},{"key":"10517_CR76","doi-asserted-by":"publisher","unstructured":"Ho CJ, Slivkins A, Suri S, Vaughan JW (2015) Incentivizing high quality crowdwork. In: International Conference on World Wide Web (WWW), WWW Conference, pp 419\u2013429. https:\/\/doi.org\/10.1145\/2736277.2741102","DOI":"10.1145\/2736277.2741102"},{"key":"10517_CR77","doi-asserted-by":"publisher","unstructured":"H\u00f6st M, Wohlin C, Thelin T (2005) Experimental context classification. In: International Conference on Software Engineering (ICSE), ACM, pp 470\u2013478. https:\/\/doi.org\/10.1145\/1062455.1062539","DOI":"10.1145\/1062455.1062539"},{"key":"10517_CR78","doi-asserted-by":"publisher","unstructured":"Huang Y, Leach K, Sharafi Z, McKay N, Santander T, Weimer W (2020) Biases and differences in code review using medical imaging and eye-tracking: genders, humans, and machines. In: Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC\/FSE), ACM, pp 456\u2013468. https:\/\/doi.org\/10.1145\/3368089.3409681","DOI":"10.1145\/3368089.3409681"},{"key":"10517_CR79","doi-asserted-by":"publisher","unstructured":"Jolak R, Savary-Leblanc M, Dalibor M, Wortmann A, Hebig R, Vincur J, Pol\u00e1sek I, Pallec XL, G\u00e9rard S, Chaudron MRV (2020) Software engineering whispers: the effect of textual vs. graphical software design descriptions on software design communication. Empirical Software Engineering 25(6):4427\u20134471. https:\/\/doi.org\/10.1007\/s10664-020-09835-6","DOI":"10.1007\/s10664-020-09835-6"},{"key":"10517_CR80","doi-asserted-by":"publisher","first-page":"106559","DOI":"10.1016\/j.infsof.2021.106559","volume":"134","author":"M J\u00f8rgensen","year":"2021","unstructured":"J\u00f8rgensen M, Grov J (2021) A field experiment on trialsourcing and the effect of contract types on outsourced software development. Inf Softw Technol 134:106559. https:\/\/doi.org\/10.1016\/j.infsof.2021.106559","journal-title":"Inf Softw Technol"},{"issue":"12","key":"10517_CR81","doi-asserted-by":"publisher","first-page":"2892","DOI":"10.1109\/TSE.2020.2973638","volume":"47","author":"M J\u00f8rgensen","year":"2021","unstructured":"J\u00f8rgensen M, Bergersen GR, Liest\u00f8l K (2021) Relations between effort estimates, skill indicators, and measured programming skill. IEEE Trans Software Eng 47(12):2892\u20132906. https:\/\/doi.org\/10.1109\/TSE.2020.2973638","journal-title":"IEEE Trans Software Eng"},{"key":"10517_CR82","doi-asserted-by":"publisher","DOI":"10.1007\/978-1-4757-3304-4","author":"N Juristo","year":"2001","unstructured":"Juristo N, Moreno AM (2001) Basics of software engineering experimentation. Springer. https:\/\/doi.org\/10.1007\/978-1-4757-3304-4","journal-title":"Springer"},{"key":"10517_CR83","doi-asserted-by":"publisher","unstructured":"Kafali \u00d6, Ajmeri N, Singh MP (2020) DESEN: Specification of sociotechnical systems via patterns of regulation and control. ACM Transactions on Software Engineering and Methodology, 29(1):7:1\u20137:50. https:\/\/doi.org\/10.1145\/3365664","DOI":"10.1145\/3365664"},{"issue":"2","key":"10517_CR84","doi-asserted-by":"publisher","first-page":"461","DOI":"10.1523\/JNEUROSCI.1583-10.2011","volume":"31","author":"MJ Kang","year":"2011","unstructured":"Kang MJ, Rangel A, Camus M, Camerer CF (2011) Hypothetical and real choice differentially activate common valuation areas. J Neurosci 31(2):461\u2013468","journal-title":"J Neurosci"},{"issue":"7","key":"10517_CR85","doi-asserted-by":"publisher","first-page":"1315","DOI":"10.1109\/TSE.2019.2920377","volume":"47","author":"I Karac","year":"2021","unstructured":"Karac I, Turhan B, Juristo N (2021) A controlled experiment with novice developers on the impact of task description granularity on software quality in test-driven development. IEEE Trans Software Eng 47(7):1315\u20131330. https:\/\/doi.org\/10.1109\/TSE.2019.2920377","journal-title":"IEEE Trans Software Eng"},{"key":"10517_CR86","doi-asserted-by":"publisher","unstructured":"Karras O, Schneider K, Fricker SA (2020) Representing software project vision by means of video: a quality model for vision videos. J Syst Softw 162. https:\/\/doi.org\/10.1016\/j.jss.2019.110479","DOI":"10.1016\/j.jss.2019.110479"},{"key":"10517_CR87","doi-asserted-by":"publisher","unstructured":"Kettles D, St\u00a0Louis R, Steinbart P (2017) An experimental investigation of the individual and joint effects of financial and non-financial lnc entives on knowledge sharing using enterprise social media. Communications of the Association for Information Systems, 41(1):639\u2013673. https:\/\/doi.org\/10.17705\/1CAIS.04127","DOI":"10.17705\/1CAIS.04127"},{"issue":"3","key":"10517_CR88","doi-asserted-by":"publisher","first-page":"183","DOI":"10.1007\/s11301-014-0111-y","volume":"65","author":"F Keusch","year":"2015","unstructured":"Keusch F (2015) Why do people participate in web surveys? Applying survey participation theory to internet survey data collection. Management Review Quarterly 65(3):183\u2013216","journal-title":"Management Review Quarterly"},{"key":"10517_CR89","doi-asserted-by":"publisher","first-page":"106635","DOI":"10.1016\/j.infsof.2021.106635","volume":"138","author":"FM Kifetew","year":"2021","unstructured":"Kifetew FM, Perini A, Susi A, Siena A, Mu\u00f1ante D, Morales-Ramirez I (2021) Automating user-feedback driven requirements prioritization. Inf Softw Technol 138:106635. https:\/\/doi.org\/10.1016\/j.infsof.2021.106635","journal-title":"Inf Softw Technol"},{"key":"10517_CR90","doi-asserted-by":"publisher","unstructured":"Kirby LJ, Boerstra E, Anderson ZJC, Rubin J (2021) Weighing the evidence: on relationship types in microservice extraction. In: International Conference on Program Comprehension (ICPC), IEEE, pp 358\u2013368. https:\/\/doi.org\/10.1109\/ICPC52881.2021.00041","DOI":"10.1109\/ICPC52881.2021.00041"},{"key":"10517_CR91","doi-asserted-by":"publisher","DOI":"10.4135\/9781483384733","author":"RE Kirk","year":"2013","unstructured":"Kirk RE (2013) Experimental design: procedures for the behavioral sciences. Sage. https:\/\/doi.org\/10.4135\/9781483384733","journal-title":"Sage"},{"key":"10517_CR92","doi-asserted-by":"publisher","DOI":"10.1201\/b19467","author":"BA Kitchenham","year":"2015","unstructured":"Kitchenham BA, Budgen D, Brereton OP (2015) Evidence-Based Software Engineering and Systematic Reviews. CRC Press. https:\/\/doi.org\/10.1201\/b19467","journal-title":"CRC Press"},{"issue":"1","key":"10517_CR93","doi-asserted-by":"publisher","first-page":"110","DOI":"10.1007\/s10664-013-9279-3","volume":"20","author":"AJ Ko","year":"2015","unstructured":"Ko AJ, LaToza TD, Burnett MM (2015) A practical guide to controlled experiments of software engineering tools with human participants. Empir Softw Eng 20(1):110\u2013141. https:\/\/doi.org\/10.1007\/s10664-013-9279-3","journal-title":"Empir Softw Eng"},{"key":"10517_CR94","doi-asserted-by":"crossref","unstructured":"Krishnamurthy S, Tripathi AK (2006) Bounty programs in free\/libre\/open source software. In: The Economics of Open Source Software Development, Elsevier, pp 165\u2013183","DOI":"10.1016\/B978-044452769-1\/50008-1"},{"issue":"3","key":"10517_CR95","doi-asserted-by":"publisher","first-page":"213","DOI":"10.1002\/acp.2350050305","volume":"5","author":"JA Krosnick","year":"1991","unstructured":"Krosnick JA (1991) Response strategies for coping with the cognitive demands of attitude measures in surveys. Appl Cogn Psychol 5(3):213\u2013236","journal-title":"Appl Cogn Psychol"},{"key":"10517_CR96","doi-asserted-by":"publisher","unstructured":"Krueger R, Huang Y, Liu X, Santander T, Weimer W, Leach K (2020) Neurological divide: an fMRI study of prose and code writing. In: International Conference on Software Engineering (ICSE), ACM, pp 678\u2013690. https:\/\/doi.org\/10.1145\/3377811.3380348","DOI":"10.1145\/3377811.3380348"},{"key":"10517_CR97","doi-asserted-by":"publisher","unstructured":"Kr\u00fcger J, Lausberger C, von Nostitz-Wallwitz I, Saake G, Leich T (2020) Search. review. repeat? An empirical study of threats to replicating SLR searches. Empirical Software Engineering, 25(1):627\u2013677. https:\/\/doi.org\/10.1007\/s10664-019-09763-0","DOI":"10.1007\/s10664-019-09763-0"},{"key":"10517_CR98","doi-asserted-by":"publisher","unstructured":"Kr\u00fcger J, \u00c7al\u0131kl\u0131 G, Bershadskyy D, Heyer R, Zabel S, Otto S (2022) Registered report: a laboratory experiment on using different financial-incentivization Sshemes in software-engineering experimentation. CoRR pp 1\u201310. https:\/\/doi.org\/10.48550\/arXiv.2202.10985","DOI":"10.48550\/arXiv.2202.10985"},{"key":"10517_CR99","doi-asserted-by":"publisher","first-page":"106633","DOI":"10.1016\/j.infsof.2021.106633","volume":"138","author":"SK Kuttal","year":"2021","unstructured":"Kuttal SK, Chen X, Wang Z, Balali S, Sarma A (2021) Visual resume: exploring developers\u2019 online contributions for hiring. Inf Softw Technol 138:106633. https:\/\/doi.org\/10.1016\/j.infsof.2021.106633","journal-title":"Inf Softw Technol"},{"issue":"4","key":"10517_CR100","doi-asserted-by":"publisher","first-page":"2416","DOI":"10.1007\/s10664-020-09810-1","volume":"25","author":"TD LaToza","year":"2020","unstructured":"LaToza TD, Arab M, Loksa D, Ko AJ (2020) Explicit programming strategies. Empir Softw Eng 25(4):2416\u20132449. https:\/\/doi.org\/10.1007\/s10664-020-09810-1","journal-title":"Explicit programming strategies. Empir Softw Eng"},{"key":"10517_CR101","doi-asserted-by":"publisher","first-page":"106592","DOI":"10.1016\/j.infsof.2021.106592","volume":"136","author":"A Lavalle","year":"2021","unstructured":"Lavalle A, Mat\u00e9 A, Trujillo J, Teruel MA, Rizzi S (2021) A methodology to automatically translate user requirements into visualizations: experimental validation. Inf Softw Technol 136:106592. https:\/\/doi.org\/10.1016\/j.infsof.2021.106592","journal-title":"Inf Softw Technol"},{"key":"10517_CR102","doi-asserted-by":"publisher","unstructured":"Lian X, Liu W, Zhang L (2020) Assisting engineers extracting requirements on components from domain documents. Inf Softw Technol 118. https:\/\/doi.org\/10.1016\/j.infsof.2019.106196","DOI":"10.1016\/j.infsof.2019.106196"},{"key":"10517_CR103","doi-asserted-by":"publisher","first-page":"106503","DOI":"10.1016\/j.infsof.2020.106503","volume":"133","author":"S Liu","year":"2021","unstructured":"Liu S, Li H, Jiang Z, Li X, Liu F, Zhong Y (2021) Rigorous code review by reverse engineering. Inf Softw Technol 133:106503. https:\/\/doi.org\/10.1016\/j.infsof.2020.106503","journal-title":"Inf Softw Technol"},{"issue":"4","key":"10517_CR104","doi-asserted-by":"publisher","first-page":"277","DOI":"10.1037\/mot0000116","volume":"5","author":"EA Locke","year":"2019","unstructured":"Locke EA, Schattke K (2019) Intrinsic and extrinsic motivation: time for expansion and clarification. Motivation Science 5(4):277\u2013290. https:\/\/doi.org\/10.1037\/mot0000116","journal-title":"Motivation Science"},{"issue":"4","key":"10517_CR105","doi-asserted-by":"publisher","first-page":"959","DOI":"10.1111\/j.1467-6494.2005.00335.x","volume":"73","author":"B Marcus","year":"2005","unstructured":"Marcus B, Sch\u00fctz A (2005) Who are the people reluctant to participate in research? Personality correlates of four different types of nonresponse as inferred from self-and observer ratings. J Pers 73(4):959\u2013984","journal-title":"J Pers"},{"key":"10517_CR106","doi-asserted-by":"crossref","unstructured":"Mason W, Watts DJ (2009) Financial incentives and the \u201cperformance of crowds\". In: Workshop on Human Computation (HCOMP), ACM, pp 77\u201385","DOI":"10.1145\/1600150.1600175"},{"issue":"6","key":"10517_CR107","doi-asserted-by":"publisher","first-page":"4962","DOI":"10.1007\/s10664-020-09876-x","volume":"25","author":"Z Masood","year":"2020","unstructured":"Masood Z, Hoda R, Blincoe K (2020) How agile teams make self-assignment work: a grounded theory study. Empir Softw Eng 25(6):4962\u20135005. https:\/\/doi.org\/10.1007\/s10664-020-09876-x","journal-title":"Empir Softw Eng"},{"key":"10517_CR108","doi-asserted-by":"publisher","unstructured":"McChesney IR, Bond RR (2020) Observations on the linear order of program code reading patterns in programmers with dyslexia. In: International Conference on Evaluation and Assessment in Software Engineering (EASE), ACM, pp 81\u201389. https:\/\/doi.org\/10.1145\/3383219.3383228","DOI":"10.1145\/3383219.3383228"},{"issue":"11","key":"10517_CR109","doi-asserted-by":"publisher","first-page":"2563","DOI":"10.1109\/TSE.2019.2956919","volume":"47","author":"L Melo","year":"2021","unstructured":"Melo L, Wiese I, d\u2019Amorim M (2021) Using docker to assist Q &A forum users. IEEE Trans Software Eng 47(11):2563\u20132574. https:\/\/doi.org\/10.1109\/TSE.2019.2956919","journal-title":"IEEE Trans Software Eng"},{"issue":"5","key":"10517_CR110","first-page":"1413","volume":"82","author":"A Merlo","year":"1992","unstructured":"Merlo A, Schotter A (1992) Theory and misbehavior of first-price auctions: comment. Am Econ Rev 82(5):1413\u20131425","journal-title":"Am Econ Rev"},{"issue":"9","key":"10517_CR111","doi-asserted-by":"publisher","first-page":"1872","DOI":"10.1109\/TSE.2019.2938525","volume":"47","author":"AN Meyer","year":"2021","unstructured":"Meyer AN, Murphy GC, Zimmermann T, Fritz T (2021) Enabling good work habits in software developers through reflective goal-setting. IEEE Trans Software Eng 47(9):1872\u20131885. https:\/\/doi.org\/10.1109\/TSE.2019.2938525","journal-title":"IEEE Trans Software Eng"},{"issue":"5","key":"10517_CR112","doi-asserted-by":"publisher","first-page":"936","DOI":"10.1109\/TSE.2019.2909033","volume":"47","author":"R Mohanani","year":"2021","unstructured":"Mohanani R, Turhan B, Ralph P (2021) Requirements framing affects design creativity. IEEE Trans Software Eng 47(5):936\u2013947. https:\/\/doi.org\/10.1109\/TSE.2019.2909033","journal-title":"IEEE Trans Software Eng"},{"issue":"3","key":"10517_CR113","doi-asserted-by":"publisher","first-page":"542","DOI":"10.1257\/aer.91.3.542","volume":"91","author":"B Moldovanu","year":"2001","unstructured":"Moldovanu B, Sela A (2001) The optimal allocation of prizes in contests. Am Econ Rev 91(3):542\u2013558","journal-title":"Am Econ Rev"},{"key":"10517_CR114","doi-asserted-by":"publisher","unstructured":"Morales R, Khomh F, Antoniol G (2020) RePOR: mimicking humans on refactoring tasks. Are we there yet? Empirical Software Engineering 25(4):2960\u20132996. https:\/\/doi.org\/10.1007\/s10664-020-09826-7","DOI":"10.1007\/s10664-020-09826-7"},{"issue":"10","key":"10517_CR115","doi-asserted-by":"publisher","first-page":"2225","DOI":"10.1109\/TSE.2019.2946148","volume":"47","author":"P Muntean","year":"2021","unstructured":"Muntean P, Monperrus M, Sun H, Grossklags J, Eckert C (2021) IntRepair: informed repairing of integer overflows. IEEE Trans Software Eng 47(10):2225\u20132241. https:\/\/doi.org\/10.1109\/TSE.2019.2946148","journal-title":"IEEE Trans Software Eng"},{"issue":"49","key":"10517_CR116","doi-asserted-by":"publisher","first-page":"20911","DOI":"10.1073\/pnas.1013305107","volume":"107","author":"K Murayama","year":"2010","unstructured":"Murayama K, Matsumoto M, Izuma K, Matsumoto K (2010) Neural basis of the undermining effect of monetary reward on intrinsic motivation. Proc Natl Acad Sci 107(49):20911\u201320916","journal-title":"Proc Natl Acad Sci"},{"key":"10517_CR117","doi-asserted-by":"publisher","unstructured":"Nafi KW, Roy B, Roy CK, Schneider KA (2020) A universal cross language software similarity detector for open source software categorization. J Syst Softw 162. https:\/\/doi.org\/10.1016\/j.jss.2019.110491","DOI":"10.1016\/j.jss.2019.110491"},{"key":"10517_CR118","doi-asserted-by":"publisher","unstructured":"N\u00fa\u00f1ez M, Bonhaure D, Gonz\u00e1lez M, Cernuzzi L (2020) A model-driven approach for the development of native mobile applications focusing on the data layer. J Syst Softw 161. https:\/\/doi.org\/10.1016\/j.jss.2019.110489","DOI":"10.1016\/j.jss.2019.110489"},{"issue":"5","key":"10517_CR119","doi-asserted-by":"publisher","first-page":"105","DOI":"10.1007\/s10664-021-09998-w","volume":"26","author":"J Olsson","year":"2021","unstructured":"Olsson J, Risfelt E, Besker T, Martini A, Torkar R (2021) Measuring affective states from technical debt. Empir Softw Eng 26(5):105. https:\/\/doi.org\/10.1007\/s10664-021-09998-w","journal-title":"Empir Softw Eng"},{"key":"10517_CR120","doi-asserted-by":"publisher","unstructured":"Ore J, Detweiler C, Elbaum SG (2021) An empirical study on type annotations: accuracy, speed, and suggestion effectiveness. ACM Transactions on Software Engineering and Methodology 30(2):20:1\u201320:29. https:\/\/doi.org\/10.1145\/3439775","DOI":"10.1145\/3439775"},{"key":"10517_CR121","doi-asserted-by":"publisher","unstructured":"Paltenghi M, Pradel M (2021) Thinking like a developer? comparing the attention of humans with neural models of code. In: International Conference on Automated Software Engineering (ASE), IEEE, pp 867\u2013879. https:\/\/doi.org\/10.1109\/ASE51524.2021.9678712","DOI":"10.1109\/ASE51524.2021.9678712"},{"issue":"1","key":"10517_CR122","doi-asserted-by":"publisher","first-page":"130","DOI":"10.1109\/TSE.2018.2884706","volume":"47","author":"JI Panach","year":"2021","unstructured":"Panach JI, Dieste O, Mar\u00edn B, Espa\u00f1a S, Vegas S, Pastor O, Juristo N (2021) Evaluating model-driven development claims with respect to quality: a family of experiments. IEEE Trans Software Eng 47(1):130\u2013145. https:\/\/doi.org\/10.1109\/TSE.2018.2884706","journal-title":"IEEE Trans Software Eng"},{"issue":"3","key":"10517_CR123","doi-asserted-by":"publisher","first-page":"292","DOI":"10.1111\/1467-9280.00454","volume":"13","author":"JE Parco","year":"2002","unstructured":"Parco JE, Rapoport A, Stein WE (2002) Effects of financial incentives on the breakdown of mutual trust. Psychol Sci 13(3):292\u2013297","journal-title":"Psychol Sci"},{"key":"10517_CR124","doi-asserted-by":"publisher","DOI":"10.1016\/j.jss.2021.110987","volume":"178","author":"P Paulweber","year":"2021","unstructured":"Paulweber P, Simhandl G, Zdun U (2021) On the understandability of language constructs to structure the state and behavior in abstract state machine specifications: a controlled experiment. J Syst Softw 178:110987. https:\/\/doi.org\/10.1016\/j.jss.2021.110987","journal-title":"J Syst Softw"},{"key":"10517_CR125","doi-asserted-by":"publisher","unstructured":"Paulweber P, Simhandl G, Zdun U (2021b) Specifying with interface and trait abstractions in abstract state machines: a controlled experiment. ACM Transactions on Software Engineering and Methodology 30(4):47:1\u201347:29. https:\/\/doi.org\/10.1145\/3450968","DOI":"10.1145\/3450968"},{"key":"10517_CR126","doi-asserted-by":"publisher","unstructured":"Peitek N, Siegmund J, Apel S (2020a) What drives the reading order of programmers? an eye tracking study. In: International Conference on Program Comprehension (ICPC), ACM, pp 342\u2013353. https:\/\/doi.org\/10.1145\/3387904.3389279","DOI":"10.1145\/3387904.3389279"},{"issue":"4","key":"10517_CR127","doi-asserted-by":"publisher","first-page":"442","DOI":"10.1109\/TSE.2018.2863303","volume":"46","author":"N Peitek","year":"2020","unstructured":"Peitek N, Siegmund J, Apel S, K\u00e4stner C, Parnin C, Bethmann A, Leich T, Saake G, Brechmann A (2020) A look into programmers\u2019 heads. IEEE Trans Software Eng 46(4):442\u2013462. https:\/\/doi.org\/10.1109\/TSE.2018.2863303","journal-title":"IEEE Trans Software Eng"},{"key":"10517_CR128","doi-asserted-by":"publisher","unstructured":"Peitek N, Apel S, Parnin C, Brechmann A, Siegmund J (2021) Program comprehension and code complexity metrics: an fMRI study. In: International Conference on Software Engineering (ICSE), IEEE, pp 524\u2013536. https:\/\/doi.org\/10.1109\/ICSE43902.2021.00056","DOI":"10.1109\/ICSE43902.2021.00056"},{"key":"10517_CR129","doi-asserted-by":"publisher","unstructured":"Petersen K, Gencel C (2013) Worldviews, research methods, and their relationship to validity in empirical software engineering research. In: Joint Conference of the International Workshop on Software Measurement (IWSM) and the International Conference on Software Process and Product Measurement (Mensura), IEEE, pp 81\u201389. https:\/\/doi.org\/10.1109\/iwsm-mensura.2013.22","DOI":"10.1109\/iwsm-mensura.2013.22"},{"key":"10517_CR130","doi-asserted-by":"crossref","unstructured":"Petersen K, Wohlin C (2009) Context in industrial software engineering research. In: International Symposium on Empirical Software Engineering and Measurement (ESEM), IEEE, pp 401\u2013404","DOI":"10.1109\/ESEM.2009.5316010"},{"key":"10517_CR131","volume-title":"Incentives: GESIS Survey Guidelines","author":"K Pforr","year":"2015","unstructured":"Pforr K (2015) Incentives: GESIS Survey Guidelines. Tech. rep, Leibniz Institute for the Social Sciences"},{"key":"10517_CR132","doi-asserted-by":"crossref","unstructured":"Ralph P (2021) ACM SIGSOFT empirical standards released. ACM SIGSOFT Software Engineering Notes 46(1)","DOI":"10.1145\/3437479.3437483"},{"issue":"1","key":"10517_CR133","doi-asserted-by":"publisher","first-page":"51","DOI":"10.1109\/TSE.2018.2842188","volume":"46","author":"M Rao","year":"2020","unstructured":"Rao M, Bacon DF, Parkes DC, Seltzer MI (2020) Incentivizing deep fixes in software economies. IEEE Trans Software Eng 46(1):51\u201370. https:\/\/doi.org\/10.1109\/TSE.2018.2842188","journal-title":"IEEE Trans Software Eng"},{"key":"10517_CR134","doi-asserted-by":"publisher","unstructured":"Ren R, Castro JW, Santos A, P\u00e9rez-Soler S, Acu\u00f1a ST, de\u00a0Lara J (2020) Collaborative modelling: chatbots or on-line tools? an experimental study. In: international conference on evaluation and assessment in software engineering (EASE), ACM, pp 260\u2013269. https:\/\/doi.org\/10.1145\/3383219.3383246","DOI":"10.1145\/3383219.3383246"},{"issue":"1","key":"10517_CR135","doi-asserted-by":"publisher","first-page":"71","DOI":"10.1109\/TSE.2018.2842781","volume":"46","author":"S Romano","year":"2020","unstructured":"Romano S, Vendome C, Scanniello G, Poshyvanyk D (2020) A multi-study investigation into dead code. IEEE Trans Software Eng 46(1):71\u201399. https:\/\/doi.org\/10.1109\/TSE.2018.2842781","journal-title":"IEEE Trans Software Eng"},{"issue":"3","key":"10517_CR136","doi-asserted-by":"publisher","first-page":"315","DOI":"10.1016\/j.econlet.2004.04.020","volume":"85","author":"O Rydval","year":"2004","unstructured":"Rydval O, Ortmann A (2004) How financial incentives and cognitive abilities affect task performance in laboratory settings: an illustration. Econ Lett 85(3):315\u2013320","journal-title":"Econ Lett"},{"issue":"3","key":"10517_CR137","doi-asserted-by":"publisher","first-page":"449","DOI":"10.1007\/s11019-022-10092-1","volume":"25","author":"J R\u00f3\u017cy\u0144ska","year":"2022","unstructured":"R\u00f3\u017cy\u0144ska J (2022) The ethical anatomy of payment for research participants. Med Health Care Philos 25(3):449\u2013464","journal-title":"Med Health Care Philos"},{"issue":"6","key":"10517_CR138","doi-asserted-by":"publisher","first-page":"4759","DOI":"10.1007\/s10664-020-09865-0","volume":"25","author":"W Said","year":"2020","unstructured":"Said W, Quante J, Koschke R (2020) Mining understandable state machine models from embedded code. Empir Softw Eng 25(6):4759\u20134804. https:\/\/doi.org\/10.1007\/s10664-020-09865-0","journal-title":"Empir Softw Eng"},{"issue":"3","key":"10517_CR139","doi-asserted-by":"publisher","first-page":"42","DOI":"10.1007\/s10664-020-09895-8","volume":"26","author":"A Santos","year":"2021","unstructured":"Santos A, Vegas S, Dieste O, Uyaguari F, Tosun A, Fucci D, Turhan B, Scanniello G, Romano S, Karac I, Kuhrmann M, Mandic V, Ramac R, Pfahl D, Engblom C, Kyykka J, Rungi K, Palomeque C, Spisak J, Oivo M, Juristo N (2021) A family of experiments on test-driven development. Empir Softw Eng 26(3):42. https:\/\/doi.org\/10.1007\/s10664-020-09895-8","journal-title":"Empir Softw Eng"},{"key":"10517_CR140","doi-asserted-by":"publisher","first-page":"106407","DOI":"10.1016\/j.infsof.2020.106407","volume":"129","author":"TRD Saputri","year":"2021","unstructured":"Saputri TRD, Lee S (2021) Integrated framework for incorporating sustainability design in software engineering life-cycle: an empirical study. Inf Softw Technol 129:106407. https:\/\/doi.org\/10.1016\/j.infsof.2020.106407","journal-title":"Inf Softw Technol"},{"key":"10517_CR141","doi-asserted-by":"publisher","unstructured":"Satterfield C, Fritz T, Murphy GC (2020) Identifying and describing information seeking tasks. In: International Conference on Automated Software Engineering (ASE), IEEE, pp 797\u2013808. https:\/\/doi.org\/10.1145\/3324884.3416537","DOI":"10.1145\/3324884.3416537"},{"issue":"2","key":"10517_CR142","doi-asserted-by":"publisher","first-page":"1259","DOI":"10.1007\/s10664-019-09790-x","volume":"25","author":"M Sayagh","year":"2020","unstructured":"Sayagh M, Kerzazi N, Petrillo F, Bennani K, Adams B (2020) What should your run-time configuration framework do to help developers? Empir Softw Eng 25(2):1259\u20131293. https:\/\/doi.org\/10.1007\/s10664-019-09790-x","journal-title":"Empir Softw Eng"},{"issue":"3","key":"10517_CR143","doi-asserted-by":"publisher","first-page":"595","DOI":"10.1109\/TSE.2019.2901468","volume":"47","author":"S Scalabrino","year":"2021","unstructured":"Scalabrino S, Bavota G, Vendome C, Linares-V\u00e1squez M, Poshyvanyk D, Oliveto R (2021) Automatically assessing code understandability. IEEE Trans Software Eng 47(3):595\u2013613. https:\/\/doi.org\/10.1109\/TSE.2019.2901468","journal-title":"IEEE Trans Software Eng"},{"issue":"2","key":"10517_CR144","doi-asserted-by":"publisher","first-page":"225","DOI":"10.1080\/13501780500086081","volume":"12","author":"A Schram","year":"2005","unstructured":"Schram A (2005) Artificiality: the tension between internal and external validity in economic experiments. Journal of Economic Methodology 12(2):225\u2013237","journal-title":"Journal of Economic Methodology"},{"key":"10517_CR145","doi-asserted-by":"publisher","DOI":"10.4337\/9781788110563","author":"A Schram","year":"2019","unstructured":"Schram A, Ule A (2019) Handbook of Research Methods and Applications in Experimental Economics. Edward Elgar Publishing. https:\/\/doi.org\/10.4337\/9781788110563","journal-title":"Edward Elgar Publishing"},{"key":"10517_CR146","doi-asserted-by":"publisher","unstructured":"Schr\u00f6ter I, Kr\u00fcger J, Siegmund J, Leich T (2017) Comprehending studies on program comprehension. In: International Conference on Program Comprehension (ICPC), IEEE, pp 308\u2013311. https:\/\/doi.org\/10.1109\/icpc.2017.9","DOI":"10.1109\/icpc.2017.9"},{"issue":"10","key":"10517_CR147","doi-asserted-by":"publisher","first-page":"2032","DOI":"10.1109\/TSE.2019.2941936","volume":"47","author":"GL Scoccia","year":"2021","unstructured":"Scoccia GL, Malavolta I, Autili M, Salle AD, Inverardi P (2021) Enhancing trustability of android applications via user-centric flexible permissions. IEEE Trans Software Eng 47(10):2032\u20132051. https:\/\/doi.org\/10.1109\/TSE.2019.2941936","journal-title":"IEEE Trans Software Eng"},{"key":"10517_CR148","doi-asserted-by":"publisher","unstructured":"Shakeel Y, Kr\u00fcger J, von Nostitz-Wallwitz I, Lausberger C, Durand GC, Saake G, Leich T (2018) (Automated) literature analysis - threats and experiences. In: International Workshop on Software Engineering for Science (SE4Science), ACM, pp 20\u201327. https:\/\/doi.org\/10.1145\/3194747.3194748","DOI":"10.1145\/3194747.3194748"},{"key":"10517_CR149","doi-asserted-by":"publisher","unstructured":"Sharafi Z, Huang Y, Leach K, Weimer W (2021) Toward an objective measure of developers\u2019 cognitive activities. ACM Transactions on Software Engineering and Methodology 30(3):30:1\u201330:40. https:\/\/doi.org\/10.1145\/3434643","DOI":"10.1145\/3434643"},{"key":"10517_CR150","doi-asserted-by":"publisher","unstructured":"Shargabi AA, Aljunid SA, Annamalai M, Zin AM (2020) Performing tasks can improve program comprehension mental model of novice developers: an empirical approach. In: International Conference on Program Comprehension (ICPC), ACM, pp 263\u2013273. https:\/\/doi.org\/10.1145\/3387904.3389277","DOI":"10.1145\/3387904.3389277"},{"key":"10517_CR151","doi-asserted-by":"publisher","unstructured":"Shaw AD, Horton JJ, Chen DL (2011) Designing incentives for inexpert human raters. In: Conference on Computer Supported Cooperative Work (CSCW), ACM, pp 275\u2013284. https:\/\/doi.org\/10.1145\/1958824.1958865","DOI":"10.1145\/1958824.1958865"},{"key":"10517_CR152","doi-asserted-by":"publisher","unstructured":"Shen Q, Wu S, Zou Y, Zhu Z, Xie B (2020) From API to NLI: a new interface for library reuse. J Syst Softw 169. https:\/\/doi.org\/10.1016\/j.jss.2020.110728","DOI":"10.1016\/j.jss.2020.110728"},{"key":"10517_CR153","doi-asserted-by":"publisher","unstructured":"Shen Q, Wu S, Zou Y, Xie B (2021) Comprehensive integration of API usage patterns. In: International Conference on Program Comprehension (ICPC), IEEE, pp 83\u201393. https:\/\/doi.org\/10.1109\/ICPC52881.2021.00017","DOI":"10.1109\/ICPC52881.2021.00017"},{"key":"10517_CR154","doi-asserted-by":"publisher","DOI":"10.1007\/978-1-84800-044-5","author":"F Shull","year":"2008","unstructured":"Shull F, Singer J, Sj\u00f8berg DIK (2008) Guide to advanced empirical software engineering. Springer. https:\/\/doi.org\/10.1007\/978-1-84800-044-5","journal-title":"Springer"},{"key":"10517_CR155","doi-asserted-by":"publisher","unstructured":"Siegmund J, Siegmund N, Apel S (2015) Views on internal and external validity in empirical software engineering. In: International Conference on Software Engineering (ICSE), IEEE, pp 9\u201319. https:\/\/doi.org\/10.1109\/icse.2015.24","DOI":"10.1109\/icse.2015.24"},{"key":"10517_CR156","unstructured":"Simmons E, Wilmot A (2004) Incentive payments on social surveys: a literature review. Survey Methodology Bulletin 53"},{"issue":"3","key":"10517_CR157","doi-asserted-by":"publisher","first-page":"49","DOI":"10.1525\/jer.2008.3.3.49","volume":"3","author":"E Singer","year":"2008","unstructured":"Singer E, Couper MP (2008) Do incentives exert undue influence on survey participation? experimental evidence. J Empir Res Hum Res Ethics 3(3):49\u201356","journal-title":"J Empir Res Hum Res Ethics"},{"issue":"1","key":"10517_CR158","doi-asserted-by":"publisher","first-page":"112","DOI":"10.1177\/0002716212458082","volume":"645","author":"E Singer","year":"2013","unstructured":"Singer E, Ye C (2013) The use and effects of incentives in surveys. Ann Am Acad Pol Soc Sci 645(1):112\u2013141","journal-title":"Ann Am Acad Pol Soc Sci"},{"key":"10517_CR159","doi-asserted-by":"crossref","unstructured":"Singer E, van Hoewyk J, Maher MP (1998) Does the payment of incentives create expectation effects? Public Opinion Quarterly pp 152\u2013164","DOI":"10.1086\/297838"},{"issue":"9","key":"10517_CR160","doi-asserted-by":"publisher","first-page":"733","DOI":"10.1109\/tse.2005.97","volume":"31","author":"DIK Sj\u00f8berg","year":"2005","unstructured":"Sj\u00f8berg DIK, Hannay JE, Hansen O, Kampenes VB, Karahasanovi\u0107 A, Liborg NK, Rekdal AC (2005) A survey of controlled experiments in software engineering. IEEE Trans Software Eng 31(9):733\u2013753. https:\/\/doi.org\/10.1109\/tse.2005.97","journal-title":"IEEE Trans Software Eng"},{"key":"10517_CR161","doi-asserted-by":"crossref","unstructured":"Sj\u00f8berg DIK, Dyb\u00e5 T, J\u00f8rgensen M (2007) The future of empirical methods in software engineering research. In: Future of Software Engineering (FOSE), IEEE, pp 358\u2013378","DOI":"10.1109\/FOSE.2007.30"},{"issue":"5","key":"10517_CR162","first-page":"923","volume":"72","author":"VL Smith","year":"1982","unstructured":"Smith VL (1982) Microeconomic systems as an experimental science. Am Econ Rev 72(5):923\u2013955","journal-title":"Am Econ Rev"},{"issue":"1","key":"10517_CR163","doi-asserted-by":"publisher","first-page":"113","DOI":"10.1257\/jep.8.1.113","volume":"8","author":"VL Smith","year":"1994","unstructured":"Smith VL (1994) Economics in the laboratory. Journal of Economic Perspectives 8(1):113\u2013131","journal-title":"Journal of Economic Perspectives"},{"issue":"12","key":"10517_CR164","doi-asserted-by":"publisher","first-page":"1294","DOI":"10.1109\/TSE.2018.2877664","volume":"46","author":"M Soltani","year":"2020","unstructured":"Soltani M, Panichella A, van Deursen A (2020) Search-based crash reproduction and its impact on debugging. IEEE Trans Software Eng 46(12):1294\u20131317. https:\/\/doi.org\/10.1109\/TSE.2018.2877664","journal-title":"IEEE Trans Software Eng"},{"key":"10517_CR165","doi-asserted-by":"publisher","unstructured":"Spadini D, \u00c7alikli G, Bacchelli A (2020) Primers or reminders? the effects of existing review comments on code review. In: International Conference on Software Engineering (ICSE), ACM, pp 1171\u20131182. https:\/\/doi.org\/10.1145\/3377811.3380385","DOI":"10.1145\/3377811.3380385"},{"key":"10517_CR166","doi-asserted-by":"publisher","unstructured":"Stapleton S, Gambhir Y, LeClair A, Eberhart Z, Weimer W, Leach K, Huang Y (2020) A human study of comprehension and code summarization. In: International Conference on Program Comprehension (ICPC), ACM, pp 2\u201313. https:\/\/doi.org\/10.1145\/3387904.3389258","DOI":"10.1145\/3387904.3389258"},{"key":"10517_CR167","doi-asserted-by":"publisher","unstructured":"Stol KJ, Fitzgerald B (2020) Guidelines for conducting software engineering research. In: Contemporary Empirical Methods in Software Engineering, Springer, pp 27\u201362. https:\/\/doi.org\/10.1007\/978-3-030-32489-6_2","DOI":"10.1007\/978-3-030-32489-6_2"},{"key":"10517_CR168","doi-asserted-by":"publisher","unstructured":"Taipalus T (2020) The effects of database complexity on SQL query formulation. J Syst Softw 165. https:\/\/doi.org\/10.1016\/j.jss.2020.110576","DOI":"10.1016\/j.jss.2020.110576"},{"key":"10517_CR169","doi-asserted-by":"publisher","first-page":"111034","DOI":"10.1016\/j.jss.2021.111034","volume":"181","author":"T Taipalus","year":"2021","unstructured":"Taipalus T, Grahn H, Ghanbari H (2021) Error messages in relational database management systems: a comparison of effectiveness, usefulness, and user confidence. J Syst Softw 181:111034. https:\/\/doi.org\/10.1016\/j.jss.2021.111034","journal-title":"J Syst Softw"},{"key":"10517_CR170","doi-asserted-by":"publisher","unstructured":"Tan SH, Li Z (2020) Collaborative bug finding for android apps. In: International Conference on Software Engineering (ICSE), ACM, pp 1335\u20131347. https:\/\/doi.org\/10.1145\/3377811.3380349","DOI":"10.1145\/3377811.3380349"},{"key":"10517_CR171","doi-asserted-by":"publisher","unstructured":"Teixeira S, Agrizzi BA, Filho JGP, Rossetto S, Pereira ISA, Costa PD, Branco AF, Martinelli RR (2020) LAURA architecture: towards a simpler way of building situation-aware and business-aware IoT applications. J Syst Softw 161. https:\/\/doi.org\/10.1016\/j.jss.2019.110494","DOI":"10.1016\/j.jss.2019.110494"},{"issue":"1","key":"10517_CR172","doi-asserted-by":"publisher","first-page":"16","DOI":"10.1037\/h0022681","volume":"73","author":"RF Thompson","year":"1966","unstructured":"Thompson RF, Spencer WA (1966) Habituation: a model phenomenon for the study of neuronal substrates of behavior. Psychol Rev 73(1):16","journal-title":"Psychol Rev"},{"issue":"11","key":"10517_CR173","doi-asserted-by":"publisher","first-page":"2438","DOI":"10.1109\/TSE.2019.2949811","volume":"47","author":"A Tosun","year":"2021","unstructured":"Tosun A, Dieste O, Vegas S, Pfahl D, Rungi K, Juristo N (2021) Investigating the impact of development task on external quality in test-driven development: an industry experiment. IEEE Trans Software Eng 47(11):2438\u20132456. https:\/\/doi.org\/10.1109\/TSE.2019.2949811","journal-title":"IEEE Trans Software Eng"},{"key":"10517_CR174","doi-asserted-by":"publisher","unstructured":"Uddin G, Khomh F, Roy CK (2020) Mining API usage scenarios from stack overflow. Inf Softw Technol 122. https:\/\/doi.org\/10.1016\/j.infsof.2020.106277","DOI":"10.1016\/j.infsof.2020.106277"},{"key":"10517_CR175","doi-asserted-by":"publisher","unstructured":"Uddin G, Khomh F, Roy CK (2021) Automatic API usage scenario documentation from technical Q &A sites. ACM Transactions on Software Engineering and Methodology 30(3):31:1\u201331:45. https:\/\/doi.org\/10.1145\/3439769","DOI":"10.1145\/3439769"},{"key":"10517_CR176","doi-asserted-by":"publisher","unstructured":"Uesbeck PM, Peterson CS, Sharif B, Stefik A (2020) A randomized controlled trial on the effects of embedded computer language switching. In: Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC\/FSE), ACM, pp 410\u2013420. https:\/\/doi.org\/10.1145\/3368089.3409701","DOI":"10.1145\/3368089.3409701"},{"key":"10517_CR177","doi-asserted-by":"publisher","unstructured":"Urbieta M, Antonelli L, Rossi G, do Prado Leite JCS, (2020) The impact of using a domain language for an agile requirements management. Inf Softw Technol 127. https:\/\/doi.org\/10.1016\/j.infsof.2020.106375","DOI":"10.1016\/j.infsof.2020.106375"},{"key":"10517_CR178","doi-asserted-by":"publisher","unstructured":"Valderas P, Torres V, Pelechano V (2020) A microservice composition approach based on the choreography of BPMN fragments. Inf Softw Technol 127. https:\/\/doi.org\/10.1016\/j.infsof.2020.106370","DOI":"10.1016\/j.infsof.2020.106370"},{"issue":"3","key":"10517_CR179","doi-asserted-by":"publisher","first-page":"2218","DOI":"10.1007\/s10664-019-09765-y","volume":"25","author":"C Vassallo","year":"2020","unstructured":"Vassallo C, Proksch S, Zemp T, Gall HC (2020) Every build you break: developer-oriented assistance for build failure resolution. Empir Softw Eng 25(3):2218\u20132257. https:\/\/doi.org\/10.1007\/s10664-019-09765-y","journal-title":"Empir Softw Eng"},{"key":"10517_CR180","doi-asserted-by":"crossref","unstructured":"Veen Fv, G\u00f6ritz AS, Sattler S (2016) Response effects of prenotification, prepaid cash, prepaid vouchers, and postpaid vouchers: an experimental comparison. Social Science Computer Review 34(3)","DOI":"10.1177\/0894439315585074"},{"issue":"4","key":"10517_CR181","doi-asserted-by":"publisher","first-page":"2844","DOI":"10.1007\/s10664-020-09805-y","volume":"25","author":"S Vegas","year":"2020","unstructured":"Vegas S, Riofr\u00edo P, Marcos E, Juristo N (2020) On (mis)perceptions of testing effectiveness: an empirical study. Empir Softw Eng 25(4):2844\u20132896. https:\/\/doi.org\/10.1007\/s10664-020-09805-y","journal-title":"Empir Softw Eng"},{"issue":"1","key":"10517_CR182","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1007\/s10664-019-09738-1","volume":"25","author":"A Viticchi\u00e9","year":"2020","unstructured":"Viticchi\u00e9 A, Regano L, Basile C, Torchiano M, Ceccato M, Tonella P (2020) Empirical assessment of the effort needed to attack programs protected with client\/server code splitting. Empir Softw Eng 25(1):1\u201348. https:\/\/doi.org\/10.1007\/s10664-019-09738-1","journal-title":"Empir Softw Eng"},{"key":"10517_CR183","unstructured":"Wang X, Sanders GL (2019) For money, and for fun: exploring the effects of gamification and financial incentives on motivating online review generation. In: Americas Conference on Information Systems (AMCIS), AIS"},{"key":"10517_CR184","doi-asserted-by":"publisher","unstructured":"Wang Y, Zhang M (2020) Reducing implicit gender biases in software development: does intergroup contact theory work? In: Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC\/FSE), ACM, pp 580\u2013592. https:\/\/doi.org\/10.1145\/3368089.3409762","DOI":"10.1145\/3368089.3409762"},{"issue":"2","key":"10517_CR185","doi-asserted-by":"publisher","first-page":"129","DOI":"10.1080\/00031305.2016.1154108","volume":"70","author":"RL Wasserstein","year":"2016","unstructured":"Wasserstein RL, Lazar NA (2016) The ASA statement on p-values: context, process, and purpose. Am Stat 70(2):129\u2013133. https:\/\/doi.org\/10.1080\/00031305.2016.1154108","journal-title":"Am Stat"},{"key":"10517_CR186","doi-asserted-by":"publisher","unstructured":"Wasserstein RL, Schirm AL, Lazar NA (2019) Moving to a world beyond \u201cp< 0.05\u201d. The American Statistician 73(sup1):1\u201319. https:\/\/doi.org\/10.1080\/00031305.2019.1583913","DOI":"10.1080\/00031305.2019.1583913"},{"issue":"3","key":"10517_CR187","doi-asserted-by":"publisher","first-page":"187","DOI":"10.1007\/s10683-006-9121-5","volume":"9","author":"RA Weber","year":"2006","unstructured":"Weber RA, Camerer CF (2006) \u201cBehavioral experiments\u2019\u2019 in economics. Exp Econ 9(3):187\u2013192. https:\/\/doi.org\/10.1007\/s10683-006-9121-5","journal-title":"Exp Econ"},{"key":"10517_CR188","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-93363-4","author":"J Weimann","year":"2019","unstructured":"Weimann J, Brosig-Koch J (2019) Methods in experimental economics. Springer. https:\/\/doi.org\/10.1007\/978-3-319-93363-4","journal-title":"Springer"},{"key":"10517_CR189","doi-asserted-by":"publisher","unstructured":"Wiese ES, Rafferty AN, Moseke G (2021) Students\u2019 misunderstanding of the order of evaluation in conjoined conditions. In: International Conference on Program Comprehension (ICPC), IEEE, pp 476\u2013484. https:\/\/doi.org\/10.1109\/ICPC52881.2021.00055","DOI":"10.1109\/ICPC52881.2021.00055"},{"key":"10517_CR190","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-29044-2","author":"C Wohlin","year":"2012","unstructured":"Wohlin C, Runeson P, H\u00f6st M, Ohlsson MC, Regnell B, Wessl\u00e9n A (2012) Experimentation in software engineering. Springer. https:\/\/doi.org\/10.1007\/978-3-642-29044-2","journal-title":"Springer"},{"key":"10517_CR191","doi-asserted-by":"publisher","unstructured":"Wyrich M, Preikschat A, Graziotin D, Wagner S (2021) The mind is a powerful place: how showing code comprehensibility metrics influences code understanding. In: International Conference on Software Engineering (ICSE), IEEE, pp 512\u2013523. https:\/\/doi.org\/10.1109\/ICSE43902.2021.00055","DOI":"10.1109\/ICSE43902.2021.00055"},{"issue":"1","key":"10517_CR192","doi-asserted-by":"publisher","first-page":"940","DOI":"10.1007\/s10664-019-09741-6","volume":"25","author":"R Yates","year":"2020","unstructured":"Yates R, Power N, Buckley J (2020) Characterizing the transfer of program comprehension in onboarding: an information-push perspective. Empir Softw Eng 25(1):940\u2013995. https:\/\/doi.org\/10.1007\/s10664-019-09741-6","journal-title":"Empir Softw Eng"},{"key":"10517_CR193","doi-asserted-by":"publisher","unstructured":"Zieris F, Prechelt L (2020) Explaining pair programming session dynamics from knowledge gaps. In: International Conference on Software Engineering (ICSE), ACM, pp 421\u2013432. https:\/\/doi.org\/10.1145\/3377811.3380925","DOI":"10.1145\/3377811.3380925"}],"container-title":["Empirical Software Engineering"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10664-024-10517-w.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10664-024-10517-w\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10664-024-10517-w.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,9,6]],"date-time":"2024-09-06T03:59:51Z","timestamp":1725595191000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10664-024-10517-w"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,8,10]]},"references-count":193,"journal-issue":{"issue":"5","published-print":{"date-parts":[[2024,9]]}},"alternative-id":["10517"],"URL":"https:\/\/doi.org\/10.1007\/s10664-024-10517-w","relation":{},"ISSN":["1382-3256","1573-7616"],"issn-type":[{"value":"1382-3256","type":"print"},{"value":"1573-7616","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,8,10]]},"assertion":[{"value":"17 June 2024","order":1,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"10 August 2024","order":2,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors have neither financial nor non-financial interests to declare.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing Interests"}},{"value":"Does not apply.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethics Approval"}}],"article-number":"135"}}