{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,21]],"date-time":"2026-02-21T18:37:45Z","timestamp":1771699065561,"version":"3.50.1"},"reference-count":86,"publisher":"Springer Science and Business Media LLC","issue":"6","license":[{"start":{"date-parts":[[2022,8,6]],"date-time":"2022-08-06T00:00:00Z","timestamp":1659744000000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2022,8,6]],"date-time":"2022-08-06T00:00:00Z","timestamp":1659744000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100015498","name":"Ministry for Innovation and Technology","doi-asserted-by":"crossref","award":["TUDFO\/47138-1\/2019-ITM"],"award-info":[{"award-number":["TUDFO\/47138-1\/2019-ITM"]}],"id":[{"id":"10.13039\/501100015498","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/501100003825","name":"Magyar Tudom\u00e1nyos Akad\u00e9mia","doi-asserted-by":"crossref","award":["J\u00e1nos Bolyai Scholarship"],"award-info":[{"award-number":["J\u00e1nos Bolyai Scholarship"]}],"id":[{"id":"10.13039\/501100003825","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/501100004895","name":"European Social Fund","doi-asserted-by":"publisher","award":["EFOP-3.6.3-VEKOP-16-2017-0002"],"award-info":[{"award-number":["EFOP-3.6.3-VEKOP-16-2017-0002"]}],"id":[{"id":"10.13039\/501100004895","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100015498","name":"Ministry for Innovation and Technology","doi-asserted-by":"crossref","award":["NKFIH-1279-2\/2020"],"award-info":[{"award-number":["NKFIH-1279-2\/2020"]}],"id":[{"id":"10.13039\/501100015498","id-type":"DOI","asserted-by":"crossref"}]},{"name":"Ministry of Innovation and the National Research, Development and Innovation Office within the framework of the Artificial Intelligence National Laboratory Programme"},{"name":"EU","award":["GINOP-2.3.2-15-2016-00037"],"award-info":[{"award-number":["GINOP-2.3.2-15-2016-00037"]}]},{"DOI":"10.13039\/501100015763","name":"University of Szeged","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100015763","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Empir Software Eng"],"published-print":{"date-parts":[[2022,11]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Tool support for automated fault localization in program debugging is limited because state-of-the-art algorithms often fail to provide efficient help to the user. They usually offer a ranked list of suspicious code elements, but the fault is not guaranteed to be found among the highest ranks. In Spectrum-Based Fault Localization (SBFL) \u2013 which uses code coverage information of test cases and their execution outcomes to calculate the ranks \u2013, the developer has to investigate several locations before finding the faulty code element. Yet, all the knowledge she a priori has or acquires during this process is not reused by the SBFL tool. There are existing approaches in which the developer interacts with the SBFL algorithm by giving feedback on the elements of the prioritized list. We propose a new approach called <jats:italic>iFL<\/jats:italic> which extends interactive approaches by exploiting contextual knowledge of the user about the next item in the ranked list (e. g., a statement), with which larger code entities (e. g., a whole function) can be repositioned in their suspiciousness. We implemented a closely related algorithm proposed by Gong <jats:italic>et al.<\/jats:italic>, called <jats:sc>Talk<\/jats:sc>. First, we evaluated <jats:italic>iFL<\/jats:italic> using simulated users, and compared the results to SBFL and <jats:sc>Talk<\/jats:sc>. Next, we introduced two types of imperfections in the simulation: user\u2019s knowledge and confidence levels. On SIR and Defects4J, results showed notable improvements in fault localization efficiency, even with strong user imperfections. We then empirically evaluated the effectiveness of the approach with real users in two sets of experiments: a quantitative evaluation of the successfulness of using <jats:italic>iFL<\/jats:italic>, and a qualitative evaluation of practical uses of the approach with experienced developers in think-aloud sessions.<\/jats:p>","DOI":"10.1007\/s10664-022-10190-x","type":"journal-article","created":{"date-parts":[[2022,8,6]],"date-time":"2022-08-06T07:14:24Z","timestamp":1659770064000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":6,"title":["Using contextual knowledge in interactive fault localization"],"prefix":"10.1007","volume":"27","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-8442-7970","authenticated-orcid":false,"given":"Ferenc","family":"Horv\u00e1th","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"\u00c1rp\u00e1d","family":"Besz\u00e9des","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"B\u00e9la","family":"Vancsics","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Gergo\u030b","family":"Balogh","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"L\u00e1szl\u00f3","family":"Vid\u00e1cs","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Tibor","family":"Gyim\u00f3thy","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2022,8,6]]},"reference":[{"key":"10190_CR1","unstructured":"iFL (2021) Research about interactive fault localization. https:\/\/interactivefaultlocalization.github.io\/. Accessed 01 April 2021"},{"issue":"11","key":"10190_CR2","doi-asserted-by":"publisher","first-page":"1780","DOI":"10.1016\/j.jss.2009.06.035","volume":"82","author":"R Abreu","year":"2009","unstructured":"Abreu R, Zoeteweij P, Golsteijn R, van Gemund AJC (2009) A practical evaluation of spectrum-based fault localization. J. Syst. Softw. 82(11):1780\u20131792","journal-title":"J. Syst. Softw."},{"key":"10190_CR3","doi-asserted-by":"crossref","unstructured":"Abreu R, Zoeteweij P, van Gemund AJC (2007) On the accuracy of spectrum-based fault localization. In: Proceedings of the Testing: Academic and Industrial Conference Practice and Research Techniques - MUTATION, pp 89\u201398","DOI":"10.1109\/TAIC.PART.2007.13"},{"issue":"2","key":"10190_CR4","doi-asserted-by":"publisher","first-page":"124","DOI":"10.1037\/0033-295X.86.2.124","volume":"86","author":"Y Anzai","year":"1979","unstructured":"Anzai Y, Simon HA (1979) The theory of learning by doing. Psychol Rev 86(2):124\u2013140. https:\/\/doi.org\/10.1037\/0033-295X.86.2.124. https:\/\/psycnet.apa.org\/record\/1979-27801-001","journal-title":"Psychol Rev"},{"key":"10190_CR5","unstructured":"Atlas (2021) Ensoft \u2014 atlas for java and c - understand code someone else wrote!. https:\/\/www.ensoftcorp.com\/atlas\/. Accessed 12 March 2021"},{"key":"10190_CR6","doi-asserted-by":"crossref","unstructured":"B Le T-D, Lo D, Le Goues C, Grunske L (2016) A learning-to-rank based fault localization approach using likely invariants. In: Proceedings of the 25th International Symposium on Software Testing and Analysis, ACM, pp 177\u2013188","DOI":"10.1145\/2931037.2931049"},{"key":"10190_CR7","doi-asserted-by":"crossref","unstructured":"Balogh G, Horv\u00e1th F, Besz\u00e9des A (2019) Poster: Aiding Java developers with interactive fault localization in Eclipse IDE. In: Proceedings of the 12th IEEE Conference on Software Testing, Verification and Validation (ICST\u201919), Posters Track, pp 371\u2013374","DOI":"10.1109\/ICST.2019.00045"},{"key":"10190_CR8","doi-asserted-by":"crossref","unstructured":"Balogh G, Schnepper Lacerda V, Horv\u00e1th F, Besz\u00e9des A (2019) iFL for Eclipse \u2013 a tool to support interactive fault localization in Eclipse IDE","DOI":"10.1109\/ICST.2019.00045"},{"key":"10190_CR9","doi-asserted-by":"crossref","unstructured":"Bandyopadhyay A, Ghosh S (2012) Tester feedback driven fault localization. In: 2012 IEEE Fifth International Conference on Software Testing, Verification and Validation, pp 41\u201350","DOI":"10.1109\/ICST.2012.84"},{"key":"10190_CR10","doi-asserted-by":"crossref","unstructured":"Baudry B, Fleurey F, Le Traon Y (2006) Improving test suites for efficient fault localization. In: 28th international conference on Software engineering. ICSE \u201906. ACM, pp 82\u201391","DOI":"10.1145\/1134285.1134299"},{"key":"10190_CR11","doi-asserted-by":"crossref","unstructured":"Caballero R, Riesco A, Silva J (2017) A survey of algorithmic debugging. ACM Comput. Surv. 50, 4","DOI":"10.1145\/3106740"},{"key":"10190_CR12","doi-asserted-by":"crossref","unstructured":"Campos J, Riboira A, Perez A, Abreu R (2012) GZoltar: an eclipse plug-in for testing and debugging. In: Proceedings of the 27th IEEE\/ACM International Conference on Automated Software Engineering - ASE 2012. ACM Press, p 378","DOI":"10.1145\/2351676.2351752"},{"issue":"1","key":"10190_CR13","doi-asserted-by":"publisher","first-page":"61","DOI":"10.1006\/IJHC.1998.0236","volume":"50","author":"CL Corritore","year":"1999","unstructured":"Corritore CL, Wiedenbeck S (1999) Mental representations of expert procedural and object-oriented programmers in a software maintenance task. International Journal of Human-Computer Studies 50(1):61\u201383. https:\/\/doi.org\/10.1006\/IJHC.1998.0236","journal-title":"International Journal of Human-Computer Studies"},{"issue":"2-3","key":"10190_CR14","doi-asserted-by":"publisher","first-page":"227","DOI":"10.1080\/07370024.1995.9667218","volume":"10","author":"SP Davies","year":"1995","unstructured":"Davies SP, Gilmore DJ, Green TRG (1995) Are Objects That Important? Effects of Expertise and Familiarity on Classification of Object-Oriented Code. Human-Computer Interaction 10(2-3):227\u2013248. https:\/\/doi.org\/10.1080\/07370024.1995.9667218","journal-title":"Human-Computer Interaction"},{"key":"10190_CR15","doi-asserted-by":"crossref","unstructured":"Do H, Elbaum S, Rothermel G (2005) Supporting controlled experimentation with testing techniques: An infrastructure and its potential impact. Emp. Softw. Eng., 10,4","DOI":"10.1007\/s10664-005-3861-2"},{"key":"10190_CR16","unstructured":"EclEmma (2021) Eclemma - java code coverage for eclipse. https:\/\/www.eclemma.org\/. Accessed 23 March 2021"},{"issue":"3","key":"10190_CR17","doi-asserted-by":"publisher","first-page":"215","DOI":"10.1037\/0033-295X.87.3.215","volume":"87","author":"KA Ericsson","year":"1980","unstructured":"Ericsson KA, Simon HA (1980) Verbal reports as data. Psychol Rev 87(3):215\u2013251. https:\/\/doi.org\/10.1037\/0033-295X.87.3.215","journal-title":"Psychol Rev"},{"issue":"1","key":"10190_CR18","doi-asserted-by":"publisher","first-page":"343","DOI":"10.1109\/TVCG.2019.2934797","volume":"26","author":"M Fan","year":"2020","unstructured":"Fan M, Wu K, Zhao J, Li Y, Wei W, Truong KN (2020) VisTA: Integrating Machine Intelligence with Visualization to Support the Investigation of Think-Aloud Sessions. IEEE Trans Vis Comput Graph 26(1):343\u2013352. https:\/\/doi.org\/10.1109\/TVCG.2019.2934797","journal-title":"IEEE Trans Vis Comput Graph"},{"issue":"5","key":"10190_CR19","doi-asserted-by":"publisher","first-page":"623","DOI":"10.2307\/2094540","volume":"43","author":"RH Franke","year":"1978","unstructured":"Franke RH, Kaul JD (1978) The hawthorne experiments: First statistical interpretation. Am Sociol Rev 43(5):623\u2013643. http:\/\/www.jstor.org\/stable\/2094540","journal-title":"Am Sociol Rev"},{"key":"10190_CR20","doi-asserted-by":"crossref","unstructured":"Fritz T, Murphy GC, Hill E (2007) Does a programmer\u2019s activity indicate knowledge of code?. In: 6th Joint Meeting of the European Software Engineering Conference and the ACM SIGSOFT Symposium on the Foundations of Software Engineering, ESEC\/FSE 2007. http:\/\/doi.acm.org\/10.1145\/1287624.1287673, pp 341\u2013350","DOI":"10.1145\/1287624.1287673"},{"key":"10190_CR21","doi-asserted-by":"crossref","unstructured":"Fritz T, Ou J, Murphy GC, Murphy-Hill E (2010) A degree-of-knowledge model to capture source code familiarity. In: Proceedings - International Conference on Software Engineering. www.nongnu.org\/cvs, vol 1, pp 385\u2013394","DOI":"10.1145\/1806799.1806856"},{"key":"10190_CR22","doi-asserted-by":"publisher","first-page":"7","DOI":"10.1145\/2635868.2635905","volume":"16-21-Nove","author":"T Fritz","year":"2014","unstructured":"Fritz T, Shepherd DC, Kevic K, Snipes W, Br\u00e4unlich C (2014) Developers\u2019 code context models for change tasks. Proceedings of the ACM SIGSOFT Symposium on the Foundations of Software Engineering 16-21-Nove:7\u201318. https:\/\/doi.org\/10.1145\/2635868.2635905","journal-title":"Proceedings of the ACM SIGSOFT Symposium on the Foundations of Software Engineering"},{"key":"10190_CR23","doi-asserted-by":"crossref","unstructured":"Fry ZP, Weimer W (2010) A human study of fault localization accuracy. In: 2010 IEEE International Conference on Software Maintenance, pp 1\u201310","DOI":"10.1109\/ICSM.2010.5609691"},{"key":"10190_CR24","unstructured":"GCOV (2021) gcov\u2014a test coverage program. https:\/\/gcc.gnu.org\/onlinedocs\/gcc\/Gcov.html, Last visited: 2020-08-13"},{"key":"10190_CR25","doi-asserted-by":"crossref","unstructured":"Gong L, Lo D, Jiang L, Zhang H (2012) Interactive fault localization leveraging simple user feedback. In: IEEE International Conference on Software Maintenance, ICSM. IEEE, pp 67\u201376","DOI":"10.1109\/ICSM.2012.6405255"},{"key":"10190_CR26","doi-asserted-by":"crossref","unstructured":"Gopstein D, Fayard AL, Apel S, Cappos J (2020) Thinking aloud about confusing code: A qualitative investigation of program comprehension and atoms of confusion. ESEC\/FSE 2020 - Proceedings of the 28th ACM Joint Meeting European Software Engineering Conference and Symposium on the Foundations of Software Engineering, pp 605\u2013616","DOI":"10.1145\/3368089.3409714"},{"issue":"5","key":"10190_CR27","doi-asserted-by":"publisher","first-page":"962","DOI":"10.1007\/s11390-009-9270-z","volume":"24","author":"D Hao","year":"2009","unstructured":"Hao D, Zhang L, Xie T, Mei H, Sun J-S (2009) Interactive Fault Localization Using Test Information. J Comput Sci Technol 24(5):962\u2013974","journal-title":"J Comput Sci Technol"},{"key":"10190_CR28","doi-asserted-by":"publisher","unstructured":"Horv\u00e1th F, Besz\u00e9des A, Vancsics B, Balogh G, Vid\u00e1cs L, Gyim\u00f3thy T Data for Experiments with Interactive Fault Localization Using Simulated and Real Users. https:\/\/doi.org\/10.5281\/zenodo.4658314","DOI":"10.5281\/zenodo.4658314"},{"key":"10190_CR29","doi-asserted-by":"crossref","unstructured":"Horv\u00e1th F, Besz\u00e9des A, Vancsics B, Balogh G, Vid\u00e1cs L, Gyim\u00f3thy T (2020) Experiments with interactive fault localization using simulated and real users. In: Proceedings of the 36th IEEE International Conference on Software Maintenance and Evolution (ICSME\u201920), pp 290\u2013300","DOI":"10.1109\/ICSME46990.2020.00036"},{"key":"10190_CR30","doi-asserted-by":"crossref","unstructured":"Horv\u00e1th F, Besz\u00e9des A, Vancsics B, Balogh G, Vid\u00e1cs L, Gyim\u00f3thy T (2020) Supplemental Material for Experiments with Interactive Fault Localization Using Simulated and Real Users, figshare","DOI":"10.1109\/ICSME46990.2020.00036"},{"key":"10190_CR31","unstructured":"iFL4Eclipse (2021) iFL 4 Eclipse. https:\/\/github.com\/InteractiveFaultLocalization\/iFL4Eclipse. Accessed 01 April 2021"},{"key":"10190_CR32","doi-asserted-by":"crossref","unstructured":"Janssen T, Abreu R, van Gemund ArjanJC (2009) Zoltar: A toolset for automatic fault localization. In: Proceedings of the 2009 IEEE\/ACM International Conference on Automated Software Engineering, IEEE Computer Society, pp 662\u2013664","DOI":"10.1109\/ASE.2009.27"},{"key":"10190_CR33","unstructured":"JavaParser (2021) Javaparser - homepage. https:\/\/javaparser.org\/ Accessed 25 October 2021"},{"key":"10190_CR34","doi-asserted-by":"crossref","unstructured":"Jones JA, Harrold MJ (2005) Empirical evaluation of the tarantula automatic fault-localization technique. In: Proc. of International Conference on Automated Software Engineering. ACM, pp 273\u2013282","DOI":"10.1145\/1101908.1101949"},{"key":"10190_CR35","doi-asserted-by":"crossref","unstructured":"Just R, Jalali D, Ernst MD (2014) Defects4j: A database of existing faults to enable controlled testing studies for java programs. In: Proceedings of the 2014 International Symposium on Software Testing and Analysis, ACM, pp 437\u2013440","DOI":"10.1145\/2610384.2628055"},{"key":"10190_CR36","doi-asserted-by":"crossref","unstructured":"Just R, Jalali D, Inozemtseva L, Ernst MD, Holmes R, Fraser G (2014) Are mutants a valid substitute for real faults in software testing?. In: Proceedings of the 22nd ACM SIGSOFT International Symposium on Foundations of Software Engineering, ACM, pp 654\u2013665","DOI":"10.1145\/2635868.2635929"},{"key":"10190_CR37","doi-asserted-by":"crossref","unstructured":"Kiss A, Hodov\u00e1n R, Gyim\u00f3thy T (2017) Coarse hierarchical delta debugging. In: Proceedings of the 2017 IEEE International Conference on Software Maintenance and Evolution (ICSME), pp 194\u2013203","DOI":"10.1109\/ICSME.2017.26"},{"key":"10190_CR38","doi-asserted-by":"crossref","unstructured":"Kochhar PS, Xia X, Lo D, Li S (2016) Practitioners\u2019 expectations on automated fault localization. In: Proceedings of the 25th International Symposium on Software Testing and Analysis - ISSTA 2016. ACM Press, New York, USA, pp 165\u2013176","DOI":"10.1145\/2931037.2931051"},{"key":"10190_CR39","doi-asserted-by":"crossref","unstructured":"Latoza TD, Garlan D, Herbsleb JD, Myers BA (2007) Program comprehension as fact finding. 6th Joint Meeting of the European Software Engineering Conference and the ACM SIGSOFT Symposium on the Foundations of Software Engineering, ESEC\/FSE 2007. 361\u2013370","DOI":"10.1145\/1287624.1287675"},{"key":"10190_CR40","doi-asserted-by":"crossref","unstructured":"Le T-DB, Thung F, Lo D (2013) Theory and Practice, Do They Match? A Case with Spectrum-Based Fault Localization. In: 2013 IEEE International Conference on Software Maintenance, pp 380\u2013383","DOI":"10.1109\/ICSM.2013.52"},{"key":"10190_CR41","doi-asserted-by":"publisher","unstructured":"Le X-BD, Bao L, Lo D, Xia X, Li S, Pasareanu C (2019) On reliability of patch correctness assessment. In: 2019 IEEE\/ACM 41st International Conference on Software Engineering (ICSE), pp 524\u2013535, DOI https:\/\/doi.org\/10.1109\/ICSE.2019.00064, (to appear in print)","DOI":"10.1109\/ICSE.2019.00064"},{"key":"10190_CR42","doi-asserted-by":"crossref","unstructured":"Lehmann D, Pradel M (2018) Feedback-directed differential testing of interactive debuggers. In: Proceedings of the 2018 26th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering. ESEC\/FSE 2018. ACM, New York, NY, USA, pp 610\u2013620","DOI":"10.1145\/3236024.3236037"},{"issue":"9","key":"10190_CR43","doi-asserted-by":"publisher","first-page":"2247","DOI":"10.1587\/transinf.E95.D.2247","volume":"E95.D","author":"Y Lei","year":"2012","unstructured":"Lei Y, Mao X, Dai Z, Wei D (2012) Effective fault localization approach using feedback. IEICE Trans Inf Syst E95.D(9):2247\u20132257","journal-title":"IEICE Trans Inf Syst"},{"key":"10190_CR44","first-page":"12","volume":"9265","author":"C Lewis","year":"1982","unstructured":"Lewis C (1982) Using the \u201cthinking Aloud\u201d Method in Cognitive Interface Design. IBM Research Report, RC-9265 9265:12. https:\/\/books.google.it\/books\/about\/Using_the_thinking_Aloud_Method_in_Cogni.html?id=F5AKHQAACAAJ&redir_esc=y","journal-title":"IBM Research Report, RC-9265"},{"key":"10190_CR45","first-page":"82","volume-title":"Iterative User-Driven Fault Localization","author":"X Li","year":"2016","unstructured":"Li X, d\u2019Amorim M, Orso A (2016) Iterative User-Driven Fault Localization. Springer International Publishing, Cham, pp 82\u201398"},{"key":"10190_CR46","doi-asserted-by":"crossref","unstructured":"Li X, Zhu S, d\u2019Amorim M, Orso A (2018) Enlightened debugging. In: Proceedings of the 40th IEEE and ACM SIGSOFT International Conference on Software Engineering (ICSE 2018). ACM","DOI":"10.1145\/3180155.3180242"},{"key":"10190_CR47","doi-asserted-by":"crossref","unstructured":"Lin Y, Sun J, Xue Y, Liu Y, Dong J (2017) Feedback-based debugging. In: Proceedings of the 39th International Conference on Software Engineering. IEEE Press, Piscataway, NJ, USA, pp 393\u2013403","DOI":"10.1109\/ICSE.2017.43"},{"key":"10190_CR48","doi-asserted-by":"crossref","unstructured":"Masri W, Assi RA (February 2014) Prevalence of coincidental correctness and mitigation of its impact on fault localization. ACM Trans. Softw. Eng. Methodol. 23(1):8:1\u20138:28","DOI":"10.1145\/2559932"},{"key":"10190_CR49","doi-asserted-by":"crossref","unstructured":"McDonald S, Edwards HM, Zhao T (2012) Exploring Think-Alouds in Usability Testing: An International Survey, vol 55","DOI":"10.1109\/TPC.2011.2182569"},{"key":"10190_CR50","doi-asserted-by":"crossref","unstructured":"N\u00f8rgaard M, Hornbaek K (2006) What Do Usability Evaluators Do in Practice?. An Explorative Study of Think-Aloud Testing. Tech. rep.","DOI":"10.1145\/1142405.1142439"},{"key":"10190_CR51","doi-asserted-by":"crossref","unstructured":"Olmsted-Hawala EL, Murphy ED, Hawala S, Ashenfelter KT (2010) Think-Aloud Protocols: A Comparison of Three Think-Aloud Protocols for use in Testing Data-Dissemination Web Sites for Usability. http:\/\/www.census.gov","DOI":"10.1145\/1753326.1753685"},{"issue":"9","key":"10190_CR52","doi-asserted-by":"publisher","first-page":"6","DOI":"10.5120\/ijca2016912206","volume":"154","author":"P Parmar","year":"2016","unstructured":"Parmar P, Patel M (2016) Software fault localization: A survey. Intl. Journal of Computer Applications 154(9):6\u201313","journal-title":"Intl. Journal of Computer Applications"},{"key":"10190_CR53","doi-asserted-by":"crossref","unstructured":"Parnin C, Orso A (2011) Are automated debugging techniques actually helping programmers?. In: Proceedings of the 2011 International Symposium on Software Testing and Analysis. ACM, pp 199\u2013209","DOI":"10.1145\/2001420.2001445"},{"key":"10190_CR54","doi-asserted-by":"crossref","unstructured":"Pearson S, Campos J, Just R, Fraser G, Abreu R, Ernst MD, Pang D, Keller B (2017) Evaluating and improving fault localization","DOI":"10.1109\/ICSE.2017.62"},{"key":"10190_CR55","doi-asserted-by":"crossref","unstructured":"Prabhakararao S, Cook C, Ruthruff J, Creswick E, Main M, Durham M, Burnett M (2003) Strategies and behaviors of end-user programmers with interactive fault localization. In: Proceedings - 2003 IEEE Symposium on Human Centric Computing Languages and Environments, HCC 2003. https:\/\/ieeexplore.ieee.org\/abstract\/document\/1260197\/, pp 15\u201322","DOI":"10.1109\/HCC.2003.1260197"},{"key":"10190_CR56","doi-asserted-by":"crossref","unstructured":"Renieris M, Reiss SP (2003) Fault localization with nearest neighbor queries. In: Proceedings of 18th IEEE International Conference on Automated Software Engineering (ASE 2003). IEEE Computer Society, pp 30\u201339","DOI":"10.1109\/ASE.2003.1240292"},{"key":"10190_CR57","doi-asserted-by":"crossref","unstructured":"Riboira A, Abreu R (2010) The GZoltar Project: A Graphical Debugger Interface. In: Testing: Academia-Industry Collaboration, Practice and Research Techniques. Springer, Berlin, Heidelberg, pp 215\u2013218","DOI":"10.1007\/978-3-642-15585-7_25"},{"issue":"12","key":"10190_CR58","doi-asserted-by":"publisher","first-page":"889","DOI":"10.1109\/TSE.2004.101","volume":"30","author":"MP Robillard","year":"2004","unstructured":"Robillard MP, Coelho W, Murphy GC (2004) How effective developers investigate source code: An exploratory study. IEEE Trans Softw Eng 30(12):889\u2013903. https:\/\/doi.org\/10.1109\/TSE.2004.101. https:\/\/ieeexplore.ieee.org\/abstract\/document\/1377187\/","journal-title":"IEEE Trans Softw Eng"},{"key":"10190_CR59","doi-asserted-by":"publisher","unstructured":"Rojas JM, Fraser G, Arcuri A (2015) Automated unit test generation during software development: a controlled experiment and think-aloud observations. In: Proceedings of the 2015 International Symposium on Software Testing and Analysis. ACM, New York, NY, USA, DOI https:\/\/doi.org\/10.1145\/2771783.2771801https:\/\/doi.org\/10.1145\/2771783.2771801, (to appear in print)","DOI":"10.1145\/2771783.2771801 10.1145\/2771783.2771801"},{"key":"10190_CR60","unstructured":"rubberduck (2021) Rubber duck debugging. https:\/\/en.wikipedia.org\/wiki\/Rubber_duck_debugging. Accessed 29 March 2021"},{"key":"10190_CR61","doi-asserted-by":"crossref","unstructured":"Schr\u00f6er M, Koschke R (March 2021) Recording, visualising and understanding developer programming behaviour. In: Proceedings of the 28th IEEE International Conference on Software Analysis, Evolution, and Reengineering (SANER\u201921)","DOI":"10.1109\/SANER50967.2021.00066"},{"key":"10190_CR62","doi-asserted-by":"crossref","unstructured":"Shamshiri S, Just R, Rojas JM, Fraser G, McMinn P, Arcuri A (2015) Do automatically generated unit tests find real faults? an empirical study of effectiveness and challenges (t). In: Automated Software Engineering (ASE), 2015 30th IEEE\/ACM International Conference on, IEEE, pp 201\u2013211","DOI":"10.1109\/ASE.2015.86"},{"key":"10190_CR63","doi-asserted-by":"crossref","unstructured":"Shynkarenko V, Zhevaho O (2020) Development of a toolkit for analyzing software debugging processes using the constructive approach. Eastern-European Journal of Enterprise Technologies, (107) pp 29\u201338","DOI":"10.15587\/1729-4061.2020.215090"},{"key":"10190_CR64","doi-asserted-by":"crossref","unstructured":"Sillito J, De Voider K, Fisher B, Murphy G (2005) Managing software change tasks: an exploratory study. pp 10","DOI":"10.1109\/ISESE.2005.1541811"},{"key":"10190_CR65","doi-asserted-by":"publisher","unstructured":"Sillito J, Murphy GC, De Volder K (2006) Questions programmers ask during software evolution tasks. In: Proceedings of the ACM SIGSOFT Symposium on the Foundations of Software Engineering, pp 23\u201334, DOI https:\/\/doi.org\/10.1145\/1181775.1181779, (to appear in print)","DOI":"10.1145\/1181775.1181779"},{"issue":"11","key":"10190_CR66","doi-asserted-by":"publisher","first-page":"976","DOI":"10.1016\/j.advengsoft.2011.05.024","volume":"42","author":"J Silva","year":"2011","unstructured":"Silva J (2011) A survey on algorithmic debugging strategies. Adv. Eng. Softw. 42(11):976\u2013991","journal-title":"Adv. Eng. Softw."},{"key":"10190_CR67","unstructured":"SoDA (2021) SoDA library. https:\/\/github.com\/sed-szeged\/soda. Accessed 13 August 2020"},{"key":"10190_CR68","doi-asserted-by":"crossref","unstructured":"Sohn J, Yoo S (2017) FLUCCS: Using code and change metrics to improve fault localization. In: Proceedings of the 26th ACM SIGSOFT International Symposium on Software Testing and Analysis. ISSTA 2017. ACM, pp 273\u2013283","DOI":"10.1145\/3092703.3092717"},{"key":"10190_CR69","unstructured":"Spinellis D (2016) Effective Debugging: 66 Specific Ways to Debug Software and Systems. Addison-Wesley Professional (1)"},{"key":"10190_CR70","doi-asserted-by":"crossref","unstructured":"Steimann F, Frenkel M, Abreu R (2013) Threats to the validity and value of empirical assessments of the accuracy of coverage-based fault locators. In: Proceedings of the 2013 International Symposium on Software Testing and Analysis. ACM, pp 314\u2013324","DOI":"10.1145\/2483760.2483767"},{"key":"10190_CR71","volume-title":"The think aloud method: A practical guide to modelling cognitive processes","author":"MW van Someren","year":"1994","unstructured":"van Someren MW, Barnard YF, Sandberg JAC (1994) The think aloud method: A practical guide to modelling cognitive processes. Academic Press, London"},{"issue":"5","key":"10190_CR72","doi-asserted-by":"publisher","first-page":"621","DOI":"10.1109\/TSMC.1986.289308","volume":"16","author":"I Vessey","year":"1986","unstructured":"Vessey I (1986) Expertise in debugging computer programs: An analysis of the content of verbal protocols. IEEE Transactions on Systems, Man, and Cybernetics 16(5):621\u2013637","journal-title":"IEEE Transactions on Systems, Man, and Cybernetics"},{"key":"10190_CR73","unstructured":"von Mayrhauser A, International AMVansProceedingsofth, 1994 U (1996) Comprehension processes during large scale maintenance. ieeexplore.ieee.org. https:\/\/ieeexplore.ieee.org\/abstract\/document\/296764\/"},{"key":"10190_CR74","unstructured":"Wallace C, Cook C, Summet J, Burnett M (2002) Assertions in End-User Software Engineering: A Think-Aloud Study. Tech. rep."},{"key":"10190_CR75","unstructured":"Wong WE, Debroy V (2009) A survey of software fault localization. Department of Computer Science, University of Texas at Dallas, Tech. Rep. UTDCS-45, 9"},{"key":"10190_CR76","doi-asserted-by":"publisher","first-page":"290","DOI":"10.1109\/TR.2013.2285319","volume":"63","author":"WE Wong","year":"2014","unstructured":"Wong WE, Debroy V, Gao R, Li Y (2014) The dstar method for effective software fault localization. IEEE Trans. Reliability 63:290\u2013308","journal-title":"IEEE Trans. Reliability"},{"issue":"8","key":"10190_CR77","doi-asserted-by":"publisher","first-page":"707","DOI":"10.1109\/TSE.2016.2521368","volume":"42","author":"WE Wong","year":"2016","unstructured":"Wong WE, Gao R, Li Y, Abreu R, Wotawa F (2016) A survey on software fault localization. IEEE Trans Softw Eng 42(8):707\u2013740","journal-title":"IEEE Trans Softw Eng"},{"key":"10190_CR78","doi-asserted-by":"crossref","unstructured":"Xia X, Bao L, Lo D, Li S (2016) \u201cAutomated Debugging Considered Harmful\u201d Considered Harmful: A User Study Revisiting the Usefulness of Spectra-Based Fault Localization Techniques with Professionals Using Real Bugs from Large Systems. In: 2016 IEEE International Conference on Software Maintenance and Evolution (ICSME). IEEE, pp 267\u2013278","DOI":"10.1109\/ICSME.2016.67"},{"issue":"4","key":"10190_CR79","doi-asserted-by":"publisher","first-page":"31:1","DOI":"10.1145\/2522920.2522924","volume":"22","author":"X Xie","year":"2013","unstructured":"Xie X, Chen TY, Kuo F-C, Xu B (2013) A theoretical analysis of the risk evaluation formulas for spectrum-based fault localization. ACM Trans. Softw. Eng. Methodol. 22(4):31:1\u201331:40","journal-title":"ACM Trans. Softw. Eng. Methodol."},{"key":"10190_CR80","doi-asserted-by":"publisher","first-page":"803","DOI":"10.1142\/S0218194011005505","volume":"21","author":"X Xu","year":"2011","unstructured":"Xu X, Debroy V, Wong WE, Guo D (2011) Ties within fault localization rankings: Exposing and addressing the problem. Int J Softw Eng Knowl Eng 21:803\u2013827","journal-title":"Int J Softw Eng Knowl Eng"},{"key":"10190_CR81","doi-asserted-by":"publisher","unstructured":"Yamashita A, Petrillo F, Khomh F, Gu\u00e9h\u00e9neuc Y-G (2018) Developer interaction traces backed by ide screen recordings from think aloud sessions. In: Proceedings of the 15th International Conference on Mining Software Repositories. MSR \u201918. Association for Computing Machinery, New York, NY, USA, pp 50\u201353, DOI https:\/\/doi.org\/10.1145\/3196398.3196457, (to appear in print)","DOI":"10.1145\/3196398.3196457"},{"issue":"3","key":"10190_CR82","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/2491509.2491513","volume":"22","author":"S Yoo","year":"2013","unstructured":"Yoo S, Harman M, Clark D (2013) Fault localization prioritization: Comparing information-theoretic and coverage-based approaches. ACM Trans Softw Eng Methodol 22(3):1","journal-title":"ACM Trans Softw Eng Methodol"},{"issue":"1","key":"10190_CR83","doi-asserted-by":"publisher","first-page":"4:1","DOI":"10.1145\/3078840","volume":"26","author":"S Yoo","year":"2017","unstructured":"Yoo S, Xie X, Kuo F-C, Chen TY, Harman M (2017) Human competitiveness of genetic programming in spectrum-based fault localisation: Theoretical and empirical analysis. ACM Trans. Softw. Eng. Methodol. 26(1):4:1\u20134:30","journal-title":"ACM Trans. Softw. Eng. Methodol."},{"key":"10190_CR84","doi-asserted-by":"crossref","unstructured":"Yu Y, Jones JA, Harrold MJ (2008) An empirical study of the effects of test-suite reduction on fault localization. In: International Conference on Software Engineering (ICSE). ACM, pp 201\u2013210","DOI":"10.1145\/1368088.1368116"},{"key":"10190_CR85","volume-title":"Why Programs Fail, Second Edition: A Guide to Systematic Debugging","author":"A Zeller","year":"2009","unstructured":"Zeller A (2009) Why Programs Fail, Second Edition: A Guide to Systematic Debugging, 2nd edn. Morgan Kaufmann Publishers Inc., San Francisco, CA, USA","edition":"2nd edn."},{"key":"10190_CR86","doi-asserted-by":"crossref","unstructured":"Zhang M, Li X, Zhang L, Khurshid S (2017) Boosting spectrum-based fault localization using pagerank. In: Proceedings of the 26th ACM SIGSOFT International Symposium on Software Testing and Analysis. ISSTA 2017. ACM, New York, NY, USA, pp 261\u2013272","DOI":"10.1145\/3092703.3092731"}],"container-title":["Empirical Software Engineering"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10664-022-10190-x.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10664-022-10190-x\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10664-022-10190-x.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,9,26]],"date-time":"2022-09-26T08:28:45Z","timestamp":1664180925000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10664-022-10190-x"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,8,6]]},"references-count":86,"journal-issue":{"issue":"6","published-print":{"date-parts":[[2022,11]]}},"alternative-id":["10190"],"URL":"https:\/\/doi.org\/10.1007\/s10664-022-10190-x","relation":{},"ISSN":["1382-3256","1573-7616"],"issn-type":[{"value":"1382-3256","type":"print"},{"value":"1573-7616","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,8,6]]},"assertion":[{"value":"26 May 2022","order":1,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"6 August 2022","order":2,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors declare that they have no conflict of interest.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"<!--Emphasis Type='Bold' removed-->Conflict of interest"}}],"article-number":"150"}}