{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,20]],"date-time":"2026-01-20T04:16:56Z","timestamp":1768882616484,"version":"3.49.0"},"reference-count":54,"publisher":"MDPI AG","issue":"1","license":[{"start":{"date-parts":[[2020,3,18]],"date-time":"2020-03-18T00:00:00Z","timestamp":1584489600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Systems"],"abstract":"<jats:p>Automation and autonomous systems are quickly becoming a more engrained aspect of modern society. The need for effective, secure computer code in a timely manner has led to the creation of automated code repair techniques to resolve issues quickly. However, the research to date has largely ignored the human factors aspects of automated code repair. The current study explored trust perceptions, reuse intentions, and trust intentions in code repair with human generated patches versus automated code repair patches. In addition, comments in the headers were manipulated to determine the effect of the presence or absence of comments in the header of the code. Participants were 51 programmers with at least 3 years\u2019 experience and knowledge of the C programming language. Results indicated only repair source (human vs. automated code repair) had a significant influence on trust perceptions and trust intentions. Specifically, participants consistently reported higher levels of perceived trustworthiness, intentions to reuse, and trust intentions for human referents compared to automated code repair. No significant effects were found for comments in the headers.<\/jats:p>","DOI":"10.3390\/systems8010008","type":"journal-article","created":{"date-parts":[[2020,3,19]],"date-time":"2020-03-19T03:54:14Z","timestamp":1584590054000},"page":"8","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":11,"title":["Would You Fix This Code for Me? Effects of Repair Source and Commenting on Trust in Code Repair"],"prefix":"10.3390","volume":"8","author":[{"given":"Gene M.","family":"Alarcon","sequence":"first","affiliation":[{"name":"Air Force Research Laboratory, Wright Patterson AFB, OH 45433, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Charles","family":"Walter","sequence":"additional","affiliation":[{"name":"Department of Computer and Information Science, University of Mississippi, University, MS 38677, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Anthony M.","family":"Gibson","sequence":"additional","affiliation":[{"name":"Consortium of Universities, Washington, DC 20036, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Rose F.","family":"Gamble","sequence":"additional","affiliation":[{"name":"Tandy School of Computer Science, University of Tulsa, Tulsa, OK 74101, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"August","family":"Capiola","sequence":"additional","affiliation":[{"name":"Air Force Research Laboratory, Wright Patterson AFB, OH 45433, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Sarah A.","family":"Jessup","sequence":"additional","affiliation":[{"name":"Air Force Research Laboratory, Wright Patterson AFB, OH 45433, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Tyler J.","family":"Ryan","sequence":"additional","affiliation":[{"name":"Department of Psychology, Wright State University, Dayton, OH 45435, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2020,3,18]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"50","DOI":"10.1518\/hfes.46.1.50.30392","article-title":"Trust in automation: Designing for appropriate reliance","volume":"46","author":"Lee","year":"2004","journal-title":"Hum. Factors"},{"key":"ref_2","unstructured":"Britton, T., Jeng, L., Carver, G., Cheak, P., and Katzenellenbogen, T. (2013). Reversible Debugging Software, Technical Report for University of Cambridge Judge Business School."},{"key":"ref_3","first-page":"19","article-title":"Software static code analysis lessons learned","volume":"16","author":"German","year":"2003","journal-title":"Crosstalk"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Arcuri, A. (2008, January 10\u201318). On the automation of fixing software bugs. Proceedings of the 30th International Conference on Software Engineering, Leipzig, Germany.","DOI":"10.1145\/1370175.1370223"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Weimer, W., Nguyen, T., Le Goues, C., and Forrest, S. (2009, January 16\u201324). Automatically finding patches using genetic programming. Proceedings of the 31st International Conference on Software Engineering, Vancouver, BC, Canada.","DOI":"10.1109\/ICSE.2009.5070536"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Gazzola, L., Mariani, L., and Micucci, D. (June, January 27). Automatic Software Repair: A Survey. Proceedings of the 2018 IEEE\/ACM 40th International Conference on Software Engineering, Gothenburg, Sweden.","DOI":"10.1145\/3180155.3182526"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"65","DOI":"10.1016\/j.jss.2019.01.069","article-title":"Astor: Exploring the design space of generate-and-validate program repair beyond GenProg","volume":"151","author":"Martinez","year":"2019","journal-title":"J. Syst. Softw."},{"key":"ref_8","unstructured":"Wickens, C.D., Li, H., Santamaria, A., Sebok, A., and Sarter, N.B. (October, January 27). Stages and levels of automation: An integrated meta-analysis. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, San Francisco, CA, USA."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"107","DOI":"10.1177\/1555343416657236","article-title":"A descriptive model of computer code trustworthiness","volume":"11","author":"Alarcon","year":"2017","journal-title":"J. Cog. Eng. Decis. Mak."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"375","DOI":"10.2307\/249649","article-title":"Reuse and productivity in integrated computer-aided software engineering: An empirical study","volume":"15","author":"Banker","year":"1991","journal-title":"MIS Q."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"23","DOI":"10.1109\/52.311048","article-title":"Effects of reuse on quality, productivity, and economics","volume":"11","author":"Lim","year":"1994","journal-title":"IEEE Softw."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Albayrak, \u00d6., and Davenport, D. (2010, January 16\u201317). Impact of maintainability defects on code inspections. Proceedings of the 2010 ACM-IEEE International Symposium on Empirical Software Engineering and Measurement, Bolzano-Bozen, Italy.","DOI":"10.1145\/1852786.1852850"},{"key":"ref_13","unstructured":"Beller, M., Bacchelli, A., Zaidman, A., and Juergens, E. (June, January 31). Modern Code Reviews in Open-Source Projects: Which Problems Do They Fix?. Proceedings of the 11th Working Conference on Mining Software Repositories, Hyderabad, India."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Alarcon, G., and Ryan, T. (2018, January 3\u20136). Trustworthiness Perceptions of Computer Code: A Heuristic-Systematic Processing Model. Proceedings of the 51st Hawaii International Conference on System Sciences, Waikoloa Village, HI, USA.","DOI":"10.24251\/HICSS.2018.671"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"752","DOI":"10.1037\/0022-3514.39.5.752","article-title":"Heuristic versus systematic information processing and the use of source versus message cues in persuasion","volume":"39","author":"Chaiken","year":"1980","journal-title":"J. Personal. Soc. Psychol."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"1389640","DOI":"10.1080\/23311908.2017.1389640","article-title":"Application of the heuristic-systematic model to computer code trustworthiness: The influence of reputation and transparency","volume":"4","author":"Alarcon","year":"2017","journal-title":"Cogent Psychol."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Capiola, A., Nelson, A.D., Walter, C., Ryan, T.J., Jessup, S.A., Alarcon, G.M., Gamble, R.F., and Pfahler, M.D. (2019, January 26\u201331). Trust in Software: Attributes of Computer Code and the Human Factors that Influence Utilization Metrics. Proceedings of the International Conference on Human-Computer Interaction, Orlando, FL, USA.","DOI":"10.1007\/978-3-030-23522-2_24"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Ryan, T.J., Walter, C., Alarcon, G.M., Gamble, R.F., Jessup, S.A., and Capiola, A.A. (2018, January 15\u201320). Individual Differences in Trust in Code: The Moderating Effects of Personality on the Trustworthiness-Trust Rrelationship. Proceedings of the International Conference on Human-Computer Interaction, Las Vegas, NV, USA.","DOI":"10.1007\/978-3-319-92270-6_53"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Walter, C., Gamble, R., Alarcon, G., Jessup, S., and Calhoun, C. (2017, January 4\u20137). Developing a mechanism to study code trustworthiness. Proceedings of the 50th Hawaii International Conference on System Sciences, Waikoloa Village, HI, USA.","DOI":"10.24251\/HICSS.2017.701"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"182","DOI":"10.1016\/j.apergo.2018.02.027","article-title":"The influence of commenting validity, placement, and style on perceptions of computer code trustworthiness: A heuristic-systematic processing approach","volume":"70","author":"Alarcon","year":"2018","journal-title":"Appl. Ergon."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Ryan, T.J., Alarcon, G.M., Walter, C., Gamble, R., Jessup, S.A., Capiola, A., and Pfahler, M.D. (2019, January 26\u201331). Trust in Automated Software Repair. Proceedings of the International Conference on Human-Computer Interaction, Orlando, FL, USA.","DOI":"10.1007\/978-3-030-22351-9_31"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"460","DOI":"10.1037\/0022-3514.66.3.460","article-title":"Heuristic processing can bias systematic processing: Effects of source credibility, argument ambiguity, and task importance on attitude judgment","volume":"66","author":"Chaiken","year":"1994","journal-title":"J. Personal. Soc. Psychol."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"277","DOI":"10.1080\/14639220500337708","article-title":"Similarities and differences between human\u2013human and human\u2013automation trust: An integrative review","volume":"8","author":"Madhavan","year":"2007","journal-title":"Theor. Issues Ergon. Sci."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"399","DOI":"10.1080\/014492999118832","article-title":"User agreement with incorrect expert system advice","volume":"18","author":"Dijkstra","year":"1999","journal-title":"Behav. Inf. Technol."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"147","DOI":"10.1207\/S15327876MP1303_2","article-title":"Predicting misuse and disuse of combat identification systems","volume":"13","author":"Dzindolet","year":"2001","journal-title":"Mil. Psychol."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"1271","DOI":"10.1109\/32.6171","article-title":"Program readability: Procedures versus comments","volume":"14","author":"Tenny","year":"1988","journal-title":"IEEE Trans. Softw. Eng."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Aman, H. (2012, January 4\u20137). An Empirical Analysis of the Impact of Comment Statements on Fault-Proneness of Small-Size Module. Proceedings of the 2012 19th Asia-Pacific Software Engineering Conference, Hong Kong, China.","DOI":"10.1109\/APSEC.2012.108"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Aman, H., Amasaki, S., Sasaki, T., and Kawahara, M. (2015, January 22\u201323). Empirical Analysis of Change-Proneness in Methods Having Local Variables with Long Names and Comments. Proceedings of the 2015 ACM\/IEEE International Symposium on Empirical Software Engineering and Measurement, Beijing, China.","DOI":"10.1109\/ESEM.2015.7321197"},{"key":"ref_29","first-page":"9","article-title":"Trust-based analysis of an Air Force collision avoidance system","volume":"24","author":"Lyons","year":"2016","journal-title":"Ergon. Des."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Aman, H., Amasaki, S., Yokogawa, T., and Kawahara, M. (2018, January 4\u20137). A Doc2Vec-Based Assessment of Comments and Its Application to Change-Prone Method Analysis. Proceedings of the 2018 25th Asia-Pacific Software Engineering Conference, Nara, Japan.","DOI":"10.1109\/APSEC.2018.00082"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"17","DOI":"10.1080\/01449290600874956","article-title":"Effect of indirect information on system trust and control allocation","volume":"27","author":"Midden","year":"2008","journal-title":"Behav. Inf. Technol."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Le, D.X.B., Bao, L., Lo, D., Xia, X., Li, S., and Pasareanu, C. (June, January 25). On Reliability of Patch Correctness Assessment. Proceedings of the 2019 IEEE\/ACM International Conference on Software Engineering, Montr\u00e9al, QC, Canada.","DOI":"10.1109\/ICSE.2019.00064"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Wang, S., Wen, M., Chen, L., Yi, X., and Mao, X. (2019, January 19\u201320). How Different is it between Machine Generated and Developer Provided Patches? An Empirical Study on the Correct Patches Generated by Automated Program Repair Techniques. Proceedings of the 2019 ACM\/IEEE International Symposium on Empirical Software Engineering and Measurement, Porto Galinhas, Brazil.","DOI":"10.1109\/ESEM.2019.8870172"},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"286","DOI":"10.1109\/3468.844354","article-title":"A model for types and levels of human interaction with automation","volume":"30","author":"Parasuraman","year":"2000","journal-title":"IEEE Trans. Syst. Man Cybern. Part. A Syst. Hum."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"13","DOI":"10.1109\/THMS.2013.2293535","article-title":"Human\u2013agent teaming for multirobot control: A review of human factors issues","volume":"44","author":"Chen","year":"2014","journal-title":"IEEE Trans. Hum.-Mach. Syst."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"377","DOI":"10.1177\/0018720816634228","article-title":"A meta-analysis of factors influencing the development of trust in automation: Implications for understanding autonomy in future systems","volume":"58","author":"Schaefer","year":"2016","journal-title":"Hum. Factors"},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"571","DOI":"10.1109\/JPROC.2011.2173265","article-title":"Moral decision making in autonomous systems: Enforcement, moral emotions, dignity, trust, and deception","volume":"100","author":"Arkin","year":"2012","journal-title":"Proc. IEEE"},{"key":"ref_38","unstructured":"Parasuraman, R., and Mouloua, S. (1996). Human Decision Makers and Automated Decision Aids. Automation and Human Performance: Theory and Applications, Lawrence Erlbaum."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"104","DOI":"10.1037\/1076-898X.6.2.104","article-title":"The dynamics of trust: Comparing humans to automation","volume":"6","author":"Lewandowsky","year":"2000","journal-title":"J. Exp. Psychol. Appl."},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Smith, E.K., Barr, E.T., Le Goues, C., and Brun, Y. (September, January 30). Is the Cure Worse Than the Disease? Overfitting in Automated Program Repair. Proceedings of the 2015 Joint Meeting on Foundations in Software Engineering, Bergamo, Italy.","DOI":"10.1145\/2786805.2786825"},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Nakajima, H., Higo, Y., Yokoyama, H., and Kusumoto, S. (2016, January 6\u20139). Toward Developer-Like Automated Program Repair\u2014Modification Comparisons between GenProg and Developers. Proceedings of the 2016 23rd Asia-Pacific Software Engineering Conference, Hamilton, New Zealand.","DOI":"10.1109\/APSEC.2016.042"},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"17","DOI":"10.1016\/0747-5632(95)00016-X","article-title":"People\u2019s perception of human and computer advice","volume":"12","author":"Waern","year":"1996","journal-title":"Comput. Hum. Behav."},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"53","DOI":"10.1207\/S15327566IJCE0401_04","article-title":"Foundations for an empirically determined scale of trust in automated systems","volume":"4","author":"Jian","year":"2000","journal-title":"Int. J. Cogn. Ergon."},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"123","DOI":"10.1037\/0021-9010.84.1.123","article-title":"The effect of the performance appraisal system on trust for management: A field quasi-experiment","volume":"84","author":"Mayer","year":"1999","journal-title":"J. Appl. Psychol."},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"1236","DOI":"10.1109\/TSE.2015.2454513","article-title":"The ManyBugs and IntroClass Benchmarks for Automated Repair of C Programs","volume":"41","author":"Holtschulte","year":"2015","journal-title":"IEEE Trans. Softw. Eng."},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"1936","DOI":"10.1007\/s10664-016-9470-4","article-title":"Automatic repair of real bugs in java: A large-scale experiment on the defects4j dataset","volume":"22","author":"Martinez","year":"2018","journal-title":"Empir. Softw. Eng."},{"key":"ref_47","unstructured":"(2020, February 27). LASER-UMASSS\/AutomatedRepairApplicabilityData. Available online: https:\/\/github.com\/LASER-UMASS\/AutomatedRepairApplicabilityData\/blob\/master\/ManyBugs.csv."},{"key":"ref_48","unstructured":"Pinheiro, J., Bates, D., DebRoy, S., Sarkar, D., and R Core Team (2019, February 06). Nlme: Linear and Nonlinear Mixed Effects Models. Available online: https:\/\/CRAN.R-project.org\/package=nlme."},{"key":"ref_49","unstructured":"R Core Team (2018). R: A language and Environment for Statistical Computing, R Foundation for Statistical Computing."},{"key":"ref_50","doi-asserted-by":"crossref","first-page":"15","DOI":"10.1037\/a0026971","article-title":"Selecting a linear mixed model for longitudinal data: Repeated measures analysis of variance, covariance pattern model, and growth curve approaches","volume":"17","author":"Liu","year":"2012","journal-title":"Psychol. Methods"},{"key":"ref_51","unstructured":"Singmann, H., Bolker, B., Westfall, J., Aust, F., and Ben-Shachar, M.S. (2019, February 06). afex: Analysis of Factorial Experiments. Available online: https:\/\/CRAN.R-project.org\/package=afex."},{"key":"ref_52","unstructured":"Herv\u00e9, M. (2019, February 06). RVAideMemoire: Testing and Plotting Procedures for Biostatistics. Available online: https:\/\/CRAN.R-project.org\/package=RVAideMemoire."},{"key":"ref_53","unstructured":"Rusnock, C.F., Miller, M.E., and Bindewald, J.M. (2017, January 20\u201323). Observations on Trust, Reliance, and Performance Measurement in Human-Automation Team Assessment. Proceedings of the 2017 Industrial and Systems Engineering Conference, Pittsburgh, PA, USA."},{"key":"ref_54","unstructured":"Parasuraman, M., and Mouloua, M. (1997). Operator Reliance on Automation: Theory and Data. Automation and Human Performance: Theory and Applications, CRC Press."}],"container-title":["Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2079-8954\/8\/1\/8\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T09:07:41Z","timestamp":1760173661000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2079-8954\/8\/1\/8"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,3,18]]},"references-count":54,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2020,3]]}},"alternative-id":["systems8010008"],"URL":"https:\/\/doi.org\/10.3390\/systems8010008","relation":{},"ISSN":["2079-8954"],"issn-type":[{"value":"2079-8954","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,3,18]]}}}