{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,3]],"date-time":"2026-06-03T07:28:06Z","timestamp":1780471686356,"version":"3.54.1"},"reference-count":89,"publisher":"Association for Computing Machinery (ACM)","issue":"7","license":[{"start":{"date-parts":[[2024,8,26]],"date-time":"2024-08-26T00:00:00Z","timestamp":1724630400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"National Science Foundation","award":["2211428"],"award-info":[{"award-number":["2211428"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Softw. Eng. Methodol."],"published-print":{"date-parts":[[2024,9,30]]},"abstract":"<jats:p>\n            Code summarization is the task of creating short, natural language descriptions of source code. It is an important part of code comprehension and a powerful method of documentation. Previous work has made progress in identifying where programmers focus in code as they write their own summaries (i.e., Writing). However, there is currently a gap in studying programmers\u2019 attention as they read code with pre-written summaries (i.e., Reading). As a result, it is currently unknown how these two forms of code comprehension compare: Reading and Writing. Also, there is a limited understanding of programmer attention with respect to program semantics. We address these shortcomings with a human eye-tracking study (\n            <jats:italic>n<\/jats:italic>\n            = 27) comparing Reading and Writing. We examined programmers\u2019 attention with respect to fine-grained program semantics, including their attention sequences (i.e., scan paths). We find distinctions in programmer attention across the comprehension tasks, similarities in reading patterns between them, and differences mediated by demographic factors. This can help guide code comprehension in both computer science education and automated code summarization. Furthermore, we mapped programmers\u2019 gaze data onto the Abstract Syntax Tree to explore another representation of human attention. We find that visual behavior on this structure is not always consistent with that on source code.\n          <\/jats:p>","DOI":"10.1145\/3664808","type":"journal-article","created":{"date-parts":[[2024,5,15]],"date-time":"2024-05-15T11:14:13Z","timestamp":1715771653000},"page":"1-37","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":10,"title":["A Tale of Two Comprehensions? Analyzing Student Programmer Attention during Code Summarization"],"prefix":"10.1145","volume":"33","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-5721-8794","authenticated-orcid":false,"given":"Zachary","family":"Karas","sequence":"first","affiliation":[{"name":"Computer Science, Vanderbilt University, Nashville, United States"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7475-7899","authenticated-orcid":false,"given":"Aakash","family":"Bansal","sequence":"additional","affiliation":[{"name":"Computer Science and Engineering, University of Notre Dame, South Bend, United States"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5719-772X","authenticated-orcid":false,"given":"Yifan","family":"Zhang","sequence":"additional","affiliation":[{"name":"Computer Science, Vanderbilt University, Nashville, United States"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7902-7625","authenticated-orcid":false,"given":"Toby","family":"Li","sequence":"additional","affiliation":[{"name":"Department of Computer Science and Engineering, University of Notre Dame, South Bend, United States"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0005-0887-1083","authenticated-orcid":false,"given":"Collin","family":"McMillan","sequence":"additional","affiliation":[{"name":"Department of Computer Science and Engineering, University of Notre Dame, South Bend, United States"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2730-5077","authenticated-orcid":false,"given":"Yu","family":"Huang","sequence":"additional","affiliation":[{"name":"Computer Science, Vanderbilt University, Nashville, United States"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2024,8,26]]},"reference":[{"key":"e_1_3_1_2_2","unstructured":"2019. Python - SDK Reference Guide - Tobii Pro SDK Documentation. Retrieved from DOI:https:\/\/developer.tobiipro.com\/python\/python-sdk-reference-guide.html"},{"key":"e_1_3_1_3_2","unstructured":"2023. Tobi Pro Fusion User Manual. Retrieved from https:\/\/go.tobii.com\/tobii-pro-fusion-user-manual"},{"issue":"01","key":"e_1_3_1_4_2","first-page":"2007","article-title":"Bonferroni and \u0160id\u00e1k corrections for multiple comparisons","volume":"3","author":"Abdi Herv\u00e9","year":"2007","unstructured":"Herv\u00e9 Abdi et\u00a0al. 2007. Bonferroni and \u0160id\u00e1k corrections for multiple comparisons. Encyclopedia of Measurement and Statistics 3, 01 (2007), 2007.","journal-title":"Encyclopedia of Measurement and Statistics"},{"key":"e_1_3_1_5_2","doi-asserted-by":"publisher","DOI":"10.1145\/3314111.3319834"},{"key":"e_1_3_1_6_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE.2019.00052"},{"key":"e_1_3_1_7_2","doi-asserted-by":"publisher","DOI":"10.1109\/VL\/HCC50065.2020.9127264"},{"key":"e_1_3_1_8_2","doi-asserted-by":"publisher","DOI":"10.1145\/3377811.3380405"},{"key":"e_1_3_1_9_2","article-title":"A transformer-based approach for source code summarization","author":"Ahmad Wasi Uddin","year":"2020","unstructured":"Wasi Uddin Ahmad, Saikat Chakraborty, Baishakhi Ray, and Kai-Wei Chang. 2020. A transformer-based approach for source code summarization. arXiv preprint arXiv:2005.00653 (2020).","journal-title":"arXiv preprint arXiv:2005.00653"},{"key":"e_1_3_1_10_2","doi-asserted-by":"publisher","DOI":"10.1016\/S0306-4573(02)00021-3"},{"key":"e_1_3_1_11_2","article-title":"Learning to represent programs with graphs","author":"Allamanis Miltiadis","year":"2017","unstructured":"Miltiadis Allamanis, Marc Brockschmidt, and Mahmoud Khademi. 2017. Learning to represent programs with graphs. arXiv preprint arXiv:1711.00740 (2017).","journal-title":"arXiv preprint arXiv:1711.00740"},{"key":"e_1_3_1_12_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2023.3279774"},{"key":"e_1_3_1_13_2","doi-asserted-by":"publisher","DOI":"10.1109\/ASE56229.2023.00092"},{"key":"e_1_3_1_14_2","doi-asserted-by":"publisher","DOI":"10.1145\/3279720.3279722"},{"key":"e_1_3_1_15_2","doi-asserted-by":"publisher","DOI":"10.1145\/1344471.1344497"},{"key":"e_1_3_1_16_2","doi-asserted-by":"publisher","DOI":"10.1145\/2363.2366"},{"key":"e_1_3_1_17_2","doi-asserted-by":"publisher","DOI":"10.3390\/s22228810"},{"key":"e_1_3_1_18_2","doi-asserted-by":"publisher","DOI":"10.1145\/3551393"},{"key":"e_1_3_1_19_2","doi-asserted-by":"publisher","DOI":"10.5555\/176313.176316"},{"key":"e_1_3_1_20_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICPC.2015.36"},{"key":"e_1_3_1_21_2","doi-asserted-by":"publisher","DOI":"10.1145\/2094131.2094133"},{"key":"e_1_3_1_22_2","doi-asserted-by":"publisher","DOI":"10.1017\/S0140525X21000017"},{"key":"e_1_3_1_23_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSM.2013.85"},{"key":"e_1_3_1_24_2","doi-asserted-by":"publisher","DOI":"10.1109\/DASC-PICom-CBDCom-CyberSciTech49142.2020.00120"},{"key":"e_1_3_1_25_2","doi-asserted-by":"publisher","DOI":"10.1145\/3030207.3030221"},{"key":"e_1_3_1_26_2","doi-asserted-by":"publisher","DOI":"10.1162\/089120105774321109"},{"key":"e_1_3_1_27_2","doi-asserted-by":"publisher","DOI":"10.3758\/BRM.42.3.692"},{"key":"e_1_3_1_28_2","first-page":"5","volume-title":"PPIG","author":"Crosby Martha E.","year":"2002","unstructured":"Martha E. Crosby, Jean Scholtz, and Susan Wiedenbeck. 2002. The roles beacons play in comprehension for novice and expert programmers. In PPIG. 5."},{"key":"e_1_3_1_29_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.scico.2012.01.004"},{"key":"e_1_3_1_30_2","doi-asserted-by":"publisher","DOI":"10.5334\/irsp.82"},{"key":"e_1_3_1_31_2","first-page":"357","volume-title":"10th IEEE International Conference on Computer Vision (ICCV\u201905) Volume 1","volume":"1","author":"Demirdjian David","year":"2005","unstructured":"David Demirdjian, Leonid Taycher, Gregory Shakhnarovich, Kristen Grauman, and Trevor Darrell. 2005. Avoiding the \u201cstreetlight effect\u201d: Tracking by exploring likelihood modes. In 10th IEEE International Conference on Computer Vision (ICCV\u201905) Volume 1, Vol. 1. IEEE, 357\u2013364."},{"key":"e_1_3_1_32_2","doi-asserted-by":"publisher","DOI":"10.1177\/105382590803100305"},{"key":"e_1_3_1_33_2","doi-asserted-by":"publisher","DOI":"10.3758\/BF03193146"},{"key":"e_1_3_1_34_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE.2017.24"},{"key":"e_1_3_1_35_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICPC.2019.00050"},{"key":"e_1_3_1_36_2","doi-asserted-by":"publisher","DOI":"10.1145\/2460999.2461003"},{"key":"e_1_3_1_37_2","doi-asserted-by":"publisher","DOI":"10.1109\/CSEE.2003.1191369"},{"key":"e_1_3_1_38_2","doi-asserted-by":"crossref","first-page":"1119","DOI":"10.1109\/ICDAR.1995.602115","volume-title":"Proceedings of 3rd International Conference on Document Analysis and Recognition","volume":"2","author":"Ha Jaekyu","year":"1995","unstructured":"Jaekyu Ha, Robert M. Haralick, and Ihsin T. Phillips. 1995. Document page decomposition by the bounding-box project. In Proceedings of 3rd International Conference on Document Analysis and Recognition, Vol. 2. IEEE, 1119\u20131122."},{"key":"e_1_3_1_39_2","doi-asserted-by":"publisher","DOI":"10.1145\/1810295.1810335"},{"key":"e_1_3_1_40_2","doi-asserted-by":"publisher","DOI":"10.1145\/3524610.3527909"},{"key":"e_1_3_1_41_2","doi-asserted-by":"publisher","DOI":"10.1145\/3209087.3209092"},{"key":"e_1_3_1_42_2","doi-asserted-by":"publisher","DOI":"10.1037\/0022-0663.84.1.76"},{"key":"e_1_3_1_43_2","doi-asserted-by":"publisher","DOI":"10.1145\/2168556.2168592"},{"key":"e_1_3_1_44_2","doi-asserted-by":"publisher","DOI":"10.1145\/3510003.3510152"},{"key":"e_1_3_1_45_2","doi-asserted-by":"publisher","DOI":"10.1145\/3368089.3409681"},{"key":"e_1_3_1_46_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE.2019.00053"},{"key":"e_1_3_1_47_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.infsof.2020.106309"},{"key":"e_1_3_1_48_2","doi-asserted-by":"crossref","unstructured":"Sarah Jessup Sasha M. Willis Gene Alarcon and Michael Lee. 2021. Using eye-tracking data to compare differences in code comprehension and code perceptions between expert and novice programmers. (2021).","DOI":"10.24251\/HICSS.2021.013"},{"key":"e_1_3_1_49_2","doi-asserted-by":"publisher","DOI":"10.1037\/0033-295X.87.4.329"},{"key":"e_1_3_1_50_2","doi-asserted-by":"publisher","DOI":"10.1145\/3468264.3468579"},{"key":"e_1_3_1_51_2","doi-asserted-by":"publisher","DOI":"10.1145\/3377811.3380348"},{"key":"e_1_3_1_52_2","doi-asserted-by":"publisher","DOI":"10.1145\/3387904.3389268"},{"key":"e_1_3_1_53_2","article-title":"Recommendations for datasets for source code summarization","author":"LeClair Alexander","year":"2019","unstructured":"Alexander LeClair and Collin McMillan. 2019. Recommendations for datasets for source code summarization. arXiv preprint arXiv:1904.02660 (2019).","journal-title":"arXiv preprint arXiv:1904.02660"},{"key":"e_1_3_1_54_2","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3359174","article-title":"Reliability and inter-rater reliability in qualitative research: Norms and guidelines for CSCW and HCI practice","volume":"3","author":"McDonald Nora","year":"2019","unstructured":"Nora McDonald, Sarita Schoenebeck, and Andrea Forte. 2019. Reliability and inter-rater reliability in qualitative research: Norms and guidelines for CSCW and HCI practice. Proceedings of the ACM on Human-Computer Interaction 3, CSCW (2019), 1\u201323.","journal-title":"Proceedings of the ACM on Human-Computer Interaction"},{"key":"e_1_3_1_55_2","doi-asserted-by":"publisher","DOI":"10.1007\/s11251-017-9445-x"},{"key":"e_1_3_1_56_2","doi-asserted-by":"publisher","DOI":"10.23919\/MIPRO.2019.8756695"},{"key":"e_1_3_1_57_2","doi-asserted-by":"publisher","DOI":"10.1521\/soco.2014.32.supp.243"},{"key":"e_1_3_1_58_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICPC.2013.6613830"},{"key":"e_1_3_1_59_2","unstructured":"Peter King Patrick Naughton Mike DeMoney Jonni Kanerva Kathy Walrath and Scott Hommel. 1999. Code conventions for the Java programming language. Sun Microsystems (1999)."},{"key":"e_1_3_1_60_2","doi-asserted-by":"publisher","DOI":"10.5555\/1146357"},{"key":"e_1_3_1_61_2","first-page":"4","article-title":"The Tobii I-VT fixation filter","volume":"21","author":"Olsen Anneli","year":"2012","unstructured":"Anneli Olsen. 2012. The Tobii I-VT fixation filter. Tobii Technology 21 (2012), 4\u201319.","journal-title":"Tobii Technology"},{"key":"e_1_3_1_62_2","first-page":"125","volume-title":"The Future of Software Engineering","author":"Parnas David Lorge","year":"2010","unstructured":"David Lorge Parnas. 2010. Precise documentation: The key to better software. In The Future of Software Engineering. Springer, 125\u2013148."},{"key":"e_1_3_1_63_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE43902.2021.00056"},{"key":"e_1_3_1_64_2","doi-asserted-by":"publisher","DOI":"10.1145\/3387904.3389279"},{"key":"e_1_3_1_65_2","doi-asserted-by":"publisher","DOI":"10.1080\/24721840.2018.1514978"},{"key":"e_1_3_1_66_2","first-page":"29","volume-title":"Proceedings of the 1st Instructional Conference on Machine Learning","volume":"242","author":"Ramos Juan","year":"2003","unstructured":"Juan Ramos et\u00a0al. 2003. Using TF-IDF to determine word relevance in document queries. In Proceedings of the 1st Instructional Conference on Machine Learning, Vol. 242. Citeseer, 29\u201348."},{"key":"e_1_3_1_67_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jss.2022.111515"},{"key":"e_1_3_1_68_2","doi-asserted-by":"publisher","DOI":"10.1037\/0033-2909.124.3.372"},{"key":"e_1_3_1_69_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2015.2442238"},{"key":"e_1_3_1_70_2","doi-asserted-by":"publisher","DOI":"10.1109\/ESEM.2015.7321188"},{"key":"e_1_3_1_71_2","doi-asserted-by":"publisher","DOI":"10.1145\/2568225.2568247"},{"key":"e_1_3_1_72_2","unstructured":"Herbert Schildt. 2014. Java: The Complete Reference. McGraw-Hill Education Group."},{"key":"e_1_3_1_73_2","doi-asserted-by":"publisher","DOI":"10.1145\/2038476.2038517"},{"key":"e_1_3_1_74_2","doi-asserted-by":"publisher","DOI":"10.1145\/2786805.2803188"},{"key":"e_1_3_1_75_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICPC.2013.6613831"},{"key":"e_1_3_1_76_2","doi-asserted-by":"publisher","DOI":"10.1109\/APSEC.2015.53"},{"key":"e_1_3_1_77_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10664-020-09829-4"},{"key":"e_1_3_1_78_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICPC.2012.6240505"},{"key":"e_1_3_1_79_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICPC.2010.41"},{"key":"e_1_3_1_80_2","article-title":"An initial study to develop an empirical test for software engineering expertise","author":"Sim Susan Elliott","year":"2006","unstructured":"Susan Elliott Sim, Sukanya Ratanotayanon, Oluwatosin Aiyelokun, and Erin Morris. 2006. An initial study to develop an empirical test for software engineering expertise. Institute for Software Research, University of California, Irvine, CA, USA, Technical Report# UCI-ISR-06-6 (2006).","journal-title":"Institute for Software Research, University of California, Irvine, CA, USA, Technical Report# UCI-ISR-06-6"},{"issue":"4","key":"e_1_3_1_81_2","first-page":"35","article-title":"Modern information retrieval: A brief overview","volume":"24","author":"Singhal Amit","year":"2001","unstructured":"Amit Singhal et\u00a0al. 2001. Modern information retrieval: A brief overview. IEEE Data Eng. Bull. 24, 4 (2001), 35\u201343.","journal-title":"IEEE Data Eng. Bull."},{"key":"e_1_3_1_82_2","first-page":"143","article-title":"Software documentation","volume":"2","author":"Sommerville Ian","year":"2001","unstructured":"Ian Sommerville. 2001. Software documentation. Software Engineering 2 (2001), 143\u2013154.","journal-title":"Software Engineering"},{"key":"e_1_3_1_83_2","doi-asserted-by":"publisher","DOI":"10.1145\/1858996.1859006"},{"key":"e_1_3_1_84_2","doi-asserted-by":"publisher","DOI":"10.1109\/ASE51524.2021.9678882"},{"key":"e_1_3_1_85_2","unstructured":"William C. Thompson. 2016. Observer effects. A Guide to Forensic DNA Profiling (2016) 171\u2013173."},{"key":"e_1_3_1_86_2","doi-asserted-by":"crossref","unstructured":"Anneliese Von Mayrhauser and A. Marie Vans. 1995. Program comprehension during software maintenance and evolution. Computer 28 8 (1995) 44\u201355.","DOI":"10.1109\/2.402076"},{"key":"e_1_3_1_87_2","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2020.3011744"},{"key":"e_1_3_1_88_2","doi-asserted-by":"publisher","DOI":"10.4324\/9781351550932-5"},{"key":"e_1_3_1_89_2","article-title":"40 years of designing code comprehension experiments: A systematic mapping study","author":"Wyrich Marvin","year":"2022","unstructured":"Marvin Wyrich, Justus Bogner, and Stefan Wagner. 2022. 40 years of designing code comprehension experiments: A systematic mapping study. arXiv preprint arXiv:2206.11102 (2022).","journal-title":"arXiv preprint arXiv:2206.11102"},{"key":"e_1_3_1_90_2","doi-asserted-by":"publisher","DOI":"10.3390\/sym14030471"}],"container-title":["ACM Transactions on Software Engineering and Methodology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3664808","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3664808","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,19]],"date-time":"2025-06-19T01:17:29Z","timestamp":1750295849000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3664808"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,8,26]]},"references-count":89,"journal-issue":{"issue":"7","published-print":{"date-parts":[[2024,9,30]]}},"alternative-id":["10.1145\/3664808"],"URL":"https:\/\/doi.org\/10.1145\/3664808","relation":{},"ISSN":["1049-331X","1557-7392"],"issn-type":[{"value":"1049-331X","type":"print"},{"value":"1557-7392","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,8,26]]},"assertion":[{"value":"2023-09-06","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-04-23","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-08-26","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}