{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,11,19]],"date-time":"2025-11-19T07:10:14Z","timestamp":1763536214578,"version":"3.37.3"},"reference-count":62,"publisher":"Springer Science and Business Media LLC","issue":"2","license":[{"start":{"date-parts":[[2024,3,4]],"date-time":"2024-03-04T00:00:00Z","timestamp":1709510400000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2024,3,4]],"date-time":"2024-03-04T00:00:00Z","timestamp":1709510400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100000781","name":"European Research Council","doi-asserted-by":"publisher","award":["646867"],"award-info":[{"award-number":["646867"]}],"id":[{"id":"10.13039\/501100000781","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100002954","name":"Universit\u00e0 degli Studi di Milano - Bicocca","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100002954","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Software Qual J"],"published-print":{"date-parts":[[2024,6]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Collecting and analyzing data about developers working on their development tasks can help improve development practices, finally increasing the productivity of teams. Indeed, monitoring and analysis tools have already been used to collect data from productivity tools. Monitoring inevitably consumes resources and, depending on their extensiveness, may significantly slow down software systems, interfering with developers\u2019 activity. There is thus a challenging trade-off between monitoring and validating applications in their operational environment and preventing the degradation of the user experience. The lack of studies about <jats:italic>when<\/jats:italic> developers perceive an overhead introduced in an application makes it\u00a0extremely difficult to fine-tune techniques working in the field. In this paper, we address this challenge by presenting an empirical study that quantifies how developers perceive overhead. The study consists of three replications of an experiment that involved 99 computer science students in total, followed by a small-scale experimental assessment of the key findings with 12 professional developers. Results show that non-negligible overhead can be introduced for a short period into applications without developers perceiving it and that the sequence in which complex operations are executed influences the perception of the system response time. This information can be exploited to design better monitoring techniques.<\/jats:p>","DOI":"10.1007\/s11219-024-09660-w","type":"journal-article","created":{"date-parts":[[2024,3,4]],"date-time":"2024-03-04T04:10:11Z","timestamp":1709525411000},"page":"567-605","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":1,"title":["A family of experiments about how developers perceive delayed system response time"],"prefix":"10.1007","volume":"32","author":[{"given":"Oscar","family":"Cornejo","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1994-8929","authenticated-orcid":false,"given":"Daniela","family":"Briola","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Daniela","family":"Micucci","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Davide","family":"Ginelli","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Leonardo","family":"Mariani","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Adri\u00e1n","family":"Santos Parrilla","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Natalia","family":"Juristo","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2024,3,4]]},"reference":[{"key":"9660_CR1","doi-asserted-by":"crossref","unstructured":"Abbas, T., Gadiraju, U., Khan, V.-J., & Markopoulos, P. (2022). Understanding user perceptions of response delays in crowd-powered conversational systems. Proc. ACM Human-Computer Interactions, 6(CSCW2).","DOI":"10.1145\/3555765"},{"key":"9660_CR2","doi-asserted-by":"publisher","unstructured":"Arapakis, I., Park, S., & Pielot, M. (2021). Impact of response latency on user behaviour in mobile web search. In: Proceedings of the 2021 Conference on Human Information Interaction and Retrieval. CHIIR \u201921, pp. 279\u2013283. Association for Computing Machinery, New York, NY, USA. https:\/\/doi.org\/10.1145\/3406522.3446038","DOI":"10.1145\/3406522.3446038"},{"issue":"1","key":"9660_CR3","first-page":"2","volume":"21","author":"M Arnold","year":"2011","unstructured":"Arnold, M., Vechev, M., & Yahav, E. (2011). QVM: An efficient runtime for detecting defects in deployed systems. ACM Transactions on Software Engineering and Methodologies, 21(1), 2\u20131235.","journal-title":"ACM Transactions on Software Engineering and Methodologies"},{"key":"9660_CR4","doi-asserted-by":"crossref","unstructured":"Baresi, L., & Ghezzi, C. (2010). The disappearing boundary between development-time and run-time. In: Proceedings of the FSE\/SDP Workshop on Future of Software Engineering Research (FoSER).","DOI":"10.1145\/1882362.1882367"},{"key":"9660_CR5","doi-asserted-by":"crossref","unstructured":"Bertolino, A., Braione, P., Angelis, G. D., Gazzola, L., Kifetew, F.M., Mariani, L., Orr\u00f9, M., Pezz\u00e8, M., Pietrantuono, R., Russo, S., & Tonella, P. (2022). A survey of field-based testing techniques. ACM Computing Surveys, 54(5).","DOI":"10.1145\/3447240"},{"key":"9660_CR6","doi-asserted-by":"crossref","unstructured":"Bartocci, E., Grosu, R., Karmarkar, A., Smolka, S. A., Stoller, S. D., & Seyster, J. N. (2012). Adaptive runtime verification. In: Proceedings of the International Conference on Runtime Verification (RV).","DOI":"10.1007\/978-3-642-35632-2_18"},{"key":"9660_CR7","doi-asserted-by":"crossref","unstructured":"Bowring, J., Orso, A., & Harrold, M. J. (2002). Monitoring deployed software using software tomography. In: Proceedings of the ACM SIGPLAN-SIGSOFT Workshop on Program Analysis for Software Tools and Engineering (PASTE).","DOI":"10.1145\/586094.586099"},{"issue":"3","key":"9660_CR8","doi-asserted-by":"publisher","first-page":"333","DOI":"10.1207\/s15327590ijhc1703_3","volume":"17","author":"I Ceaparu","year":"2004","unstructured":"Ceaparu, I., Lazar, J. N., Bessiere, K., Robinson, J., & Shneiderman, B. (2004). Determining causes and severity of end-user frustration. International Journal of Human - Computer Interaction, 17(3), 333\u2013356.","journal-title":"International Journal of Human - Computer Interaction"},{"key":"9660_CR9","doi-asserted-by":"crossref","unstructured":"Ceccato, M., Corradini, D., Gazzola, L., Kifetew, F.M., Mariani, L., Orru, M., & Tonella, P. (2020). A framework for in-vivo testing of mobile applications. In: Proceedings of the International Conference on Software Testing, Verification, and Validation (ICST).","DOI":"10.1109\/ICST46399.2020.00037"},{"key":"9660_CR10","doi-asserted-by":"crossref","unstructured":"Clause, J., & Orso, A. (2007). A technique for enabling and supporting debugging of field failures. In: Proceedings of the International Conference on Software Engineering (ICSE).","DOI":"10.1109\/ICSE.2007.10"},{"key":"9660_CR11","first-page":"69","volume-title":"What if there were no significance tests?","author":"J Cohen","year":"2016","unstructured":"Cohen, J. (2016). The earth is round (p<05). What if there were no significance tests? (pp. 69\u201382). United Kingdom: Routledge."},{"key":"9660_CR12","doi-asserted-by":"crossref","unstructured":"Colegrave, N., & Ruxton, G. D. (2017). Statistical model specification and power: recommendations on the use of test-qualified pooling in analysis of experimental data. Proceedings of the Royal Society B: Biological Sciences, 284(1851).","DOI":"10.1098\/rspb.2016.1850"},{"key":"9660_CR13","doi-asserted-by":"publisher","unstructured":"Cornejo, O., Briola, D., Micucci, D., & Mariani, L. (2017a). In the field monitoring of interactive applications. In: Proceedings of the International Conference on Software Engineering: New Ideas and Emerging Results Track (ICSE). https:\/\/doi.org\/10.1109\/ICSE-NIER.2017.19","DOI":"10.1109\/ICSE-NIER.2017.19"},{"key":"9660_CR14","doi-asserted-by":"crossref","unstructured":"Cornejo, O., Briola, D., Micucci, D., & Mariani, L. (2017b). Fragmented monitoring. In: Proceedings Second International Workshop on Pre- and Post-Deployment Verification Techniques (PrePost@iFM).","DOI":"10.4204\/EPTCS.254.5"},{"key":"9660_CR15","doi-asserted-by":"publisher","unstructured":"Cornejo, O., Briola, D., Micucci, D., & Mariani, L. (2020a) In-the-field monitoring of functional calls: Is it feasible? Journal of Systems and Software, 163(110523). https:\/\/doi.org\/10.1016\/j.jss.2020.110523","DOI":"10.1016\/j.jss.2020.110523"},{"key":"9660_CR16","doi-asserted-by":"publisher","unstructured":"Cornejo, O., Briola, D., Micucci, D., & Mariani, L. (2020b). CBR: controlled burst recording. In: Proceedings of the IEEE International Conference on Software Testing, Verification and Validation (ICST). https:\/\/doi.org\/10.1109\/ICST46399.2020.00033","DOI":"10.1109\/ICST46399.2020.00033"},{"key":"9660_CR17","doi-asserted-by":"publisher","first-page":"85913","DOI":"10.1109\/ACCESS.2019.2925855","volume":"7","author":"O Cornejo","year":"2019","unstructured":"Cornejo, O., Ginelli, D., Briola, D., Micucci, D., & Mariani, L. (2019). Field monitoring with delayed saving. IEEE Access, 7, 85913\u201385924. https:\/\/doi.org\/10.1109\/ACCESS.2019.2925855","journal-title":"IEEE Access"},{"key":"9660_CR18","doi-asserted-by":"crossref","unstructured":"Couceiro, R., Barbosa, R., Duraes, J., Duarte, G., Castelhano, J., Duarte, C., Teixeira, C., Laranjeiro, N., Medeiros, J., Carvalho, P., Castelo Branco, M., & Madeira, H. (2019). Spotting problematic code lines using nonintrusive programmers\u2019 biofeedback. In: Proceedings of the IEEE International Symposium on Software Reliability Engineering (ISSRE).","DOI":"10.1109\/ISSRE.2019.00019"},{"key":"9660_CR19","doi-asserted-by":"publisher","DOI":"10.4324\/9780203807002","volume-title":"Understanding the new statistics: Effect sizes, confidence intervals, and meta-analysis","author":"G Cumming","year":"2013","unstructured":"Cumming, G. (2013). Understanding the new statistics: Effect sizes, confidence intervals, and meta-analysis. United Kingdom: Routledge."},{"key":"9660_CR20","doi-asserted-by":"publisher","DOI":"10.1093\/acprof:oso\/9780198566427.001.0001","volume-title":"Measuring the mind: Speed, control, and age","author":"J Duncan","year":"2005","unstructured":"Duncan, J., Phillips, L., & McLeod, P. (2005). Measuring the mind: Speed, control, and age. United Kindom: Oxford Scholarship."},{"key":"9660_CR21","unstructured":"Eclipse community: Eclipse equinox weaving. https:\/\/www.eclipse.org\/equinox\/weaving\/ (visited on 2019)."},{"issue":"4","key":"9660_CR22","doi-asserted-by":"publisher","first-page":"312","DOI":"10.1109\/TSE.2005.50","volume":"31","author":"S Elbaum","year":"2005","unstructured":"Elbaum, S., & Diep, M. (2005). Profiling deployed software: Assessing strategies and testing opportunities. IEEE Transactions on Software Engineering (TSE), 31(4), 312\u2013327.","journal-title":"IEEE Transactions on Software Engineering (TSE)"},{"key":"9660_CR23","volume-title":"Discovering statistics using IBM SPSS statistics","author":"A Field","year":"2013","unstructured":"Field, A. (2013). Discovering statistics using IBM SPSS statistics. United States: Sage."},{"key":"9660_CR24","doi-asserted-by":"publisher","unstructured":"Funk, M., Cunningham, C., Kanver, D., Saikalis, C., & Pansare, R. (2020). Usable and acceptable response delays of conversational agents in automotive user interfaces. In: 12th International Conference on Automotive User Interfaces and Interactive Vehicular Applications. AutomotiveUI \u201920, pp. 262\u2013269. Association for Computing Machinery, New York, NY, USA. https:\/\/doi.org\/10.1145\/3409120.3410651","DOI":"10.1145\/3409120.3410651"},{"key":"9660_CR25","doi-asserted-by":"crossref","unstructured":"Gazzola, L., Goldstein, M., Mariani, L., Mobilio, M., Segall, I., Tundo, A., & Ussi, L. (2023). ExVivoMicroTest: Exvivo testing of microservices. Journal of Software: Evolution and Process, 35(4).","DOI":"10.1002\/smr.2452"},{"key":"9660_CR26","doi-asserted-by":"crossref","unstructured":"Gazzola, L., Mariani, L., Pastore, F., & Pezz\u00e8, M. (2017). An exploratory study of field failures. In: Proceedings of the International Symposium on Software Reliability Engineering (ISSRE).","DOI":"10.1109\/ISSRE.2017.10"},{"issue":"4","key":"9660_CR27","doi-asserted-by":"publisher","first-page":"421","DOI":"10.3102\/10769986029004421","volume":"29","author":"P Ghisletta","year":"2004","unstructured":"Ghisletta, P., & Spini, D. (2004). An introduction to generalized estimating equations and an application to assess selectivity effects in a longitudinal study on very old individuals. Journal of Educational and Behavioral Statistics, 29(4), 421\u2013437.","journal-title":"Journal of Educational and Behavioral Statistics"},{"key":"9660_CR28","unstructured":"Hirzel, M., & Chilimbi, T. (2001). Bursty tracing: A framework for low-overhead temporal profiling. In: Proceedings of the 4th ACM Workshop on Feedback-Directed and Dynamic Optimization (DYNAMO)."},{"key":"9660_CR29","doi-asserted-by":"crossref","unstructured":"Hosek, P., & Cadar, C. (2015). VARAN the unbelievable: An efficient n-version execution framework. In: Proceedings of the International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS).","DOI":"10.1145\/2694344.2694390"},{"key":"9660_CR30","unstructured":"Hoxmeier, J. A., & Cesare, C. D. (2000). System response time and user satisfaction: An experimental study of browser-based applications. In: Proceedings of the Association of Information Systems Americas Conference (AMCIS)."},{"key":"9660_CR31","doi-asserted-by":"crossref","unstructured":"Jin, G., Thakur, A., Liblit, B., & Lu, S. (2010). Instrumentation and sampling strategies for cooperative concurrency bug isolation. In: Proceedings of the ACM International Conference on Object Oriented Programming Systems Languages and Applications (OOPSLA).","DOI":"10.1145\/1869459.1869481"},{"key":"9660_CR32","doi-asserted-by":"crossref","unstructured":"Jin, W., & Orso, A. (2012). BugRedux: Reproducing field failures for in-house debugging. In: Proceedings of the International Conference on Software Engineering (ICSE).","DOI":"10.1109\/ICSE.2012.6227168"},{"key":"9660_CR33","volume-title":"Basics of software engineering experimentation","author":"N Juristo","year":"2013","unstructured":"Juristo, N., & Moreno, A. (2013). Basics of software engineering experimentation. Berlin: Springer."},{"issue":"4","key":"9660_CR34","doi-asserted-by":"publisher","first-page":"455","DOI":"10.1037\/0033-295X.94.4.455","volume":"94","author":"PR Killeen","year":"1987","unstructured":"Killeen, P. R., & Weiss, N. A. (1987). Optimal timing and the Weber function. Psychological Review, 94(4), 455\u2013468.","journal-title":"Psychological Review"},{"key":"9660_CR35","doi-asserted-by":"publisher","unstructured":"Laudato, G., Scalabrino, S., Novielli, N., Lanubile, F., & Oliveto, R. (2023). Predicting bugs by monitoring developers during task execution. In: 2023 IEEE\/ACM 45th International Conference on Software Engineering (ICSE), pp. 1\u201313. https:\/\/doi.org\/10.1109\/ICSE48619.2023.00100","DOI":"10.1109\/ICSE48619.2023.00100"},{"issue":"5","key":"9660_CR36","doi-asserted-by":"publisher","first-page":"141","DOI":"10.1145\/780822.781148","volume":"38","author":"B Liblit","year":"2003","unstructured":"Liblit, B., Aiken, A., Zheng, A. X., & Jordan, M. I. (2003). Bug isolation via remote program sampling. ACM SIGPLAN Notices, 38(5), 141\u2013154.","journal-title":"ACM SIGPLAN Notices"},{"key":"9660_CR37","doi-asserted-by":"publisher","unstructured":"Luo, C., Li, X., Liu, Y., Sakai, T., Zhang, F., Zhang, M., & Ma, S. (2017). Investigating users\u2019 time perception during web search. In: Proceedings of the 2017 Conference on Conference Human Information Interaction and Retrieval. CHIIR \u201917, pp. 127\u2013136. Association for Computing Machinery, New York, NY, USA. https:\/\/doi.org\/10.1145\/3020165.3020184","DOI":"10.1145\/3020165.3020184"},{"issue":"1","key":"9660_CR38","doi-asserted-by":"publisher","first-page":"114","DOI":"10.1037\/met0000078","volume":"22","author":"D McNeish","year":"2017","unstructured":"McNeish, D., Stapleton, L. M., & Silverman, R. D. (2017). On the unnecessary ubiquity of hierarchical linear modeling. Psychological Methods, 22(1), 114\u2013140.","journal-title":"Psychological Methods"},{"issue":"12","key":"9660_CR39","doi-asserted-by":"publisher","first-page":"1178","DOI":"10.1109\/TSE.2017.2656886","volume":"43","author":"AN Meyer","year":"2017","unstructured":"Meyer, A. N., Barton, L. E., Murphy, G. C., Zimmermann, T., & Fritz, T. (2017). The work life of developers: Activities, switches and perceived productivity. IEEE Transactions on Software Engineering (TSE), 43(12), 1178\u20131193.","journal-title":"IEEE Transactions on Software Engineering (TSE)"},{"key":"9660_CR40","doi-asserted-by":"crossref","unstructured":"Meyer, A.N., Murphy, G.C., Zimmermann, T., & Fritz, T. (2017). Design recommendations for self-monitoring in the workplace: Studies in software development. In: Proceedings of the International Conference on Human-Computer Interaction (HCI).","DOI":"10.1145\/3134714"},{"key":"9660_CR41","doi-asserted-by":"crossref","unstructured":"Miller, R. B. (1968). Response time in man-computer conversational transactions. In: Proceedings of the December 9-11, 1968, Fall Joint Computer Conference, Part I (AFIPS)","DOI":"10.1145\/1476589.1476628"},{"issue":"3","key":"9660_CR42","doi-asserted-by":"publisher","first-page":"153","DOI":"10.1080\/01449290410001669914","volume":"23","author":"FF Nah","year":"2004","unstructured":"Nah, F. F. (2004). A study on tolerable waiting time: How long are web users willing to wait? Behavior and Information Technology, 23(3), 153\u2013163.","journal-title":"Behavior and Information Technology"},{"issue":"1","key":"9660_CR43","doi-asserted-by":"publisher","first-page":"65","DOI":"10.1145\/291469.291470","volume":"42","author":"L Nielsen","year":"1999","unstructured":"Nielsen, L. (1999). User interface directions for the web. Commun. ACM, 42(1), 65\u201372.","journal-title":"Commun. ACM"},{"issue":"5","key":"9660_CR44","doi-asserted-by":"publisher","first-page":"625","DOI":"10.1007\/s10459-010-9222-y","volume":"15","author":"G Norman","year":"2010","unstructured":"Norman, G. (2010). Likert scales, levels of measurement and the laws of statistics. Advances in Health Sciences Education, 15(5), 625\u2013632.","journal-title":"Advances in Health Sciences Education"},{"key":"9660_CR45","doi-asserted-by":"crossref","unstructured":"Ohmann, P., Brown, D.B., Neelakandan, N., Linderoth, J., Liblit, B.: Optimizing customized program coverage. In: Proceedings of the International Conference on Automated Software Engineering (ASE) (2016)","DOI":"10.1145\/2970276.2970351"},{"key":"9660_CR46","doi-asserted-by":"crossref","unstructured":"Orso, A. (2010). Monitoring, analysis, and testing of deployed software. In: Proceedings of the FSE\/SDP Workshop on Future of Software Engineering Research (FoSER).","DOI":"10.1145\/1882362.1882417"},{"key":"9660_CR47","doi-asserted-by":"crossref","unstructured":"Orso, A., Liang, D., Harrold, M. J., & Lipton, R. (2002). Gamma system: Continuous evolution of software after deployment. In: Proceedings of the ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA).","DOI":"10.1145\/566172.566182"},{"key":"9660_CR48","doi-asserted-by":"crossref","unstructured":"Pavlopoulou, C., & Young, M. (1999). Residual test coverage monitoring. In: Proceedings of the International Conference on Software Engineering (ICSE).","DOI":"10.1145\/302405.302637"},{"key":"9660_CR49","doi-asserted-by":"crossref","unstructured":"Rubin, V., Lomazova, I., & van\u00a0der Aalst, W. (2014). Agile development with software process mining. In: Proceedings of the International Conference on Software and System Process (ICSSP).","DOI":"10.1145\/2600821.2600842"},{"key":"9660_CR50","volume-title":"Designing and engineering time: The psychology of time perception in software","author":"SC Seow","year":"2008","unstructured":"Seow, S. C. (2008). Designing and engineering time: The psychology of time perception in software. Boston: Addison-Wesley Professional."},{"key":"9660_CR51","doi-asserted-by":"crossref","unstructured":"Shneiderman, B. (1987). Designing the user interface strategies for effective human - computer interaction. SIGBIO Newsl, 9(1).","DOI":"10.1145\/25065.950626"},{"key":"9660_CR52","volume-title":"Designing the user interface: Strategies for effective human computer interaction","author":"B Shneiderman","year":"2009","unstructured":"Shneiderman, B., Plaisant, C., Cohen, M., & Jacobs, S. (2009). Designing the user interface: Strategies for effective human computer interaction (5th ed.). Boston: Addison-Wesley.","edition":"5"},{"key":"9660_CR53","doi-asserted-by":"publisher","unstructured":"S\u00f6derstr\u00f6m, U., B\u00e5\u00e5th, M., Mejtoft, T. (2018). The users\u2019 time perception: The effect of various animation speeds on loading screens. In: Proceedings of the 36th European Conference on Cognitive Ergonomics. ECCE \u201918. Association for Computing Machinery, New York, NY, USA. https:\/\/doi.org\/10.1145\/3232078.3232092","DOI":"10.1145\/3232078.3232092"},{"key":"9660_CR54","doi-asserted-by":"crossref","unstructured":"Song, W., Sung, N., Chun, B.-G., & Kim, J. (2014). Reducing energy consumption of smartphones using user-perceived response time analysis. In: Proceedings of the Workshop on Mobile Computing Systems and Applications (HotMobile).","DOI":"10.1145\/2565585.2565595"},{"key":"9660_CR55","doi-asserted-by":"crossref","unstructured":"Tu, S., He, X., Li, X., & Ratcliff, J. J. (2001). A systematic approach to reduction of user-perceived response time for GIS web services. In: Proceedings of the ACM International Symposium on Advances in Geographic Information Systems (SIGSPATIAL).","DOI":"10.1145\/512161.512172"},{"key":"9660_CR56","doi-asserted-by":"crossref","unstructured":"van\u00a0der Aalst, W. (2012). Process mining: Overview and opportunities. ACM Transactions on Management Information Systems (TMIS), 3(2).","DOI":"10.1145\/2229156.2229157"},{"key":"9660_CR57","doi-asserted-by":"publisher","first-page":"67","DOI":"10.1016\/j.ajp.2016.08.016","volume":"24","author":"U Wadgave","year":"2016","unstructured":"Wadgave, U., & Khairnar, M. R. (2016). Parametric tests for Likert scale: For and against. Asian journal of psychiatry, 24, 67.","journal-title":"Asian journal of psychiatry"},{"key":"9660_CR58","doi-asserted-by":"crossref","unstructured":"Wang, M. (2014). Generalized estimating equations in longitudinal data analysis: A review and recent developments. Advances in Statistics, 2014","DOI":"10.1155\/2014\/303728"},{"key":"9660_CR59","doi-asserted-by":"publisher","DOI":"10.1201\/b17198","volume-title":"Linear mixed models: A practical guide using statistical software","author":"BT West","year":"2014","unstructured":"West, B. T., Welch, K. B., & Galecki, A. T. (2014). Linear mixed models: A practical guide using statistical software. United Kingdom: Chapman and Hall\/CRC."},{"key":"9660_CR60","doi-asserted-by":"publisher","first-page":"63","DOI":"10.1007\/978-1-4612-2294-1_3","volume-title":"Linear mixed models in practice","author":"G Verbeke","year":"1997","unstructured":"Verbeke, G. (1997). Linear mixed models for longitudinal data. Linear mixed models in practice (pp. 63\u2013153). Berlin: Springer."},{"key":"9660_CR61","doi-asserted-by":"crossref","unstructured":"Zorn, C. J. (2001). Generalized estimating equation models for correlated data: A review with applications. American Journal of Political Science, pp. 470\u2013490.","DOI":"10.2307\/2669353"},{"key":"9660_CR62","doi-asserted-by":"crossref","unstructured":"Z\u00fcger, M., Corley, C., Meyer, A.N., Li, B., Fritz, T., Shepherd, D., Augustine, V., Francis, P., Kraft, N., & Snipes, W. (2017). Reducing interruptions at work: A large-scale field study of flowlight. In: Proceedings of the Conference on Human Factors in Computing Systems (CHI).","DOI":"10.1145\/3025453.3025662"}],"container-title":["Software Quality Journal"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11219-024-09660-w.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s11219-024-09660-w\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11219-024-09660-w.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,6,20]],"date-time":"2024-06-20T09:16:28Z","timestamp":1718874988000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s11219-024-09660-w"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,3,4]]},"references-count":62,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2024,6]]}},"alternative-id":["9660"],"URL":"https:\/\/doi.org\/10.1007\/s11219-024-09660-w","relation":{},"ISSN":["0963-9314","1573-1367"],"issn-type":[{"type":"print","value":"0963-9314"},{"type":"electronic","value":"1573-1367"}],"subject":[],"published":{"date-parts":[[2024,3,4]]},"assertion":[{"value":"12 January 2024","order":1,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"4 March 2024","order":2,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors declare no competing interests.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}]}}