{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,10]],"date-time":"2026-04-10T11:56:28Z","timestamp":1775822188597,"version":"3.50.1"},"publisher-location":"New York, NY, USA","reference-count":72,"publisher":"ACM","license":[{"start":{"date-parts":[[2020,7,13]],"date-time":"2020-07-13T00:00:00Z","timestamp":1594598400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/100014718","name":"National Science Foundation","doi-asserted-by":"publisher","award":["CCF-1452959, CCF-1717607, CCF-1908633, CCF-1763674"],"award-info":[{"award-number":["CCF-1452959, CCF-1717607, CCF-1908633, CCF-1763674"]}],"id":[{"id":"10.13039\/100014718","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2020,7,13]]},"DOI":"10.1145\/3387904.3389258","type":"proceedings-article","created":{"date-parts":[[2020,9,12]],"date-time":"2020-09-12T20:04:38Z","timestamp":1599941078000},"page":"2-13","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":69,"title":["A Human Study of Comprehension and Code Summarization"],"prefix":"10.1145","author":[{"given":"Sean","family":"Stapleton","sequence":"first","affiliation":[{"name":"University of Michigan"}]},{"given":"Yashmeet","family":"Gambhir","sequence":"additional","affiliation":[{"name":"University of Michigan"}]},{"given":"Alexander","family":"LeClair","sequence":"additional","affiliation":[{"name":"University of Notre Dame"}]},{"given":"Zachary","family":"Eberhart","sequence":"additional","affiliation":[{"name":"University of Notre Dame"}]},{"given":"Westley","family":"Weimer","sequence":"additional","affiliation":[{"name":"University of Michigan"}]},{"given":"Kevin","family":"Leach","sequence":"additional","affiliation":[{"name":"University of Michigan"}]},{"given":"Yu","family":"Huang","sequence":"additional","affiliation":[{"name":"University of Michigan"}]}],"member":"320","published-online":{"date-parts":[[2020,9,12]]},"reference":[{"key":"e_1_3_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1109\/CSMR.2011.24"},{"key":"e_1_3_2_1_2_1","first-page":"2091","volume-title":"International Conference on Machine Learning","author":"Allamanis M.","year":"2016","unstructured":"Allamanis , M. , Peng , H. , and Sutton , C . A convolutional attention network for extreme summarization of source code . In International Conference on Machine Learning ( 2016 ), pp. 2091 -- 2100 . Allamanis, M., Peng, H., and Sutton, C. A convolutional attention network for extreme summarization of source code. In International Conference on Machine Learning (2016), pp. 2091--2100."},{"key":"e_1_3_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11334-013-0207-x"},{"key":"e_1_3_2_1_4_1","volume-title":"Proceedings of the ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and\/or Summarization","author":"Banerjee S.","year":"2005","unstructured":"Banerjee , S. , and Lavie , A . METEOR: An automatic metric for MT evaluation with improved correlation with human judgments . In Proceedings of the ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and\/or Summarization ( Ann Arbor, Michigan , June 2005 ), Association for Computational Linguistics, pp. 65--72. Banerjee, S., and Lavie, A. METEOR: An automatic metric for MT evaluation with improved correlation with human judgments. In Proceedings of the ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and\/or Summarization (Ann Arbor, Michigan, June 2005), Association for Computational Linguistics, pp. 65--72."},{"key":"e_1_3_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICPC.2019.00033"},{"key":"e_1_3_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.5555\/800099.803210"},{"key":"e_1_3_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0020-7373(83)80031-5"},{"key":"e_1_3_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1109\/WPC.2005.22"},{"key":"e_1_3_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1145\/1390630.1390664"},{"key":"e_1_3_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1145\/1858996.1859005"},{"key":"e_1_3_2_1_11_1","first-page":"178","volume-title":"The effectiveness of source code obfuscation: an experimental assessment","author":"Ceccato M.","unstructured":"Ceccato , M. , Di Penta , M. , Nagra , J. , Falcarin , P. , Ricca , F. , Torchiano , M. , and Tonella , P . The effectiveness of source code obfuscation: an experimental assessment . pp. 178 -- 187 . Ceccato, M., Di Penta, M., Nagra, J., Falcarin, P., Ricca, F., Torchiano, M., and Tonella, P. The effectiveness of source code obfuscation: an experimental assessment. pp. 178--187."},{"key":"e_1_3_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.5555\/645544.655870"},{"key":"e_1_3_2_1_13_1","first-page":"63","volume-title":"Proceedings of MT summit IX","author":"Coughlin D.","year":"2003","unstructured":"Coughlin , D. Correlating automated and human assessments of machine translation quality . In Proceedings of MT summit IX ( 2003 ), pp. 63 -- 70 . Coughlin, D. Correlating automated and human assessments of machine translation quality. In Proceedings of MT summit IX (2003), pp. 63--70."},{"key":"e_1_3_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1145\/1085313.1085331"},{"key":"e_1_3_2_1_15_1","first-page":"3","article-title":"A comparative evaluation of program comprehension measures","volume":"52","author":"Dunsmore A.","year":"2000","unstructured":"Dunsmore , A. , and Roper , M . A comparative evaluation of program comprehension measures . The Journal of Systems and Software 52 , 3 ( 2000 ), 121--129. Dunsmore, A., and Roper, M. A comparative evaluation of program comprehension measures. The Journal of Systems and Software 52, 3 (2000), 121--129.","journal-title":"The Journal of Systems and Software"},{"key":"e_1_3_2_1_16_1","volume-title":"Measuring nominal scale agreement among many raters. Psychological bulletin 76, 5","author":"Fleiss J.L.","year":"1971","unstructured":"Fleiss , J.L. Measuring nominal scale agreement among many raters. Psychological bulletin 76, 5 ( 1971 ), 378. Fleiss, J.L. Measuring nominal scale agreement among many raters. Psychological bulletin 76, 5 (1971), 378."},{"key":"e_1_3_2_1_17_1","first-page":"175","volume-title":"International Conference on Software Engineering","author":"Floyd B.","year":"2017","unstructured":"Floyd , B. , Santander , T. , and Weimer , W . Decoding the representation of code in the brain: an fmri study of code review and expertise . In International Conference on Software Engineering ( 2017 ), pp. 175 -- 186 . Floyd, B., Santander, T., and Weimer, W. Decoding the representation of code in the brain: an fmri study of code review and expertise. In International Conference on Software Engineering (2017), pp. 175--186."},{"key":"e_1_3_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1109\/WCRE.2007.21"},{"key":"e_1_3_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2017.2664836"},{"key":"e_1_3_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1145\/2338965.2336775"},{"key":"e_1_3_2_1_21_1","unstructured":"HackerRank. HackerRank. https:\/\/www.hackerrank.com\/.  HackerRank. HackerRank. https:\/\/www.hackerrank.com\/."},{"key":"e_1_3_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1145\/1810295.1810335"},{"key":"e_1_3_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1109\/WCRE.2010.13"},{"key":"e_1_3_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE.2009.5070524"},{"key":"e_1_3_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10664-018-9621-x"},{"key":"e_1_3_2_1_26_1","volume-title":"Three approaches to qualitative content analysis. Qualitative health research 15, 9","author":"Hsieh H.-F.","year":"2005","unstructured":"Hsieh , H.-F. , and Shannon , S. E . Three approaches to qualitative content analysis. Qualitative health research 15, 9 ( 2005 ), 1277--1288. Hsieh, H.-F., and Shannon, S. E. Three approaches to qualitative content analysis. Qualitative health research 15, 9 (2005), 1277--1288."},{"key":"e_1_3_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1145\/3196321.3196334"},{"key":"e_1_3_2_1_28_1","volume-title":"Deep code comment generation with hybrid lexical and syntactical information. Empirical Software Engineering","author":"Hu X.","year":"2019","unstructured":"Hu , X. , Li , G. , Xia , X. , Lo , D. , and Jin , Z . Deep code comment generation with hybrid lexical and syntactical information. Empirical Software Engineering ( 2019 ), 1--39. Hu, X., Li, G., Xia, X., Lo, D., and Jin, Z. Deep code comment generation with hybrid lexical and syntactical information. Empirical Software Engineering (2019), 1--39."},{"key":"e_1_3_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jss.2011.09.019"},{"key":"e_1_3_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/P16-1195"},{"key":"e_1_3_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1145\/2597008.2597140"},{"key":"e_1_3_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1023\/B:LIDA.0000048322.42751.ca"},{"key":"e_1_3_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11334-007-0031-2"},{"key":"e_1_3_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE.2019.00087"},{"key":"e_1_3_2_1_35_1","volume-title":"2019 Annual Conference of the North Americal Chapter of the Association for Computational Linguistics (NAACL) (June","author":"LeClair A.","year":"2019","unstructured":"LeClair , A. , and McMillan , C. Recommendataions for datasets for source code summarization . In 2019 Annual Conference of the North Americal Chapter of the Association for Computational Linguistics (NAACL) (June 2019 ). LeClair, A., and McMillan, C. Recommendataions for datasets for source code summarization. In 2019 Annual Conference of the North Americal Chapter of the Association for Computational Linguistics (NAACL) (June 2019)."},{"key":"e_1_3_2_1_36_1","unstructured":"LeetCode LLC. LeetCode - The World's Leading Online Programming Learning Platform. https:\/\/leetcode.com\/.  LeetCode LLC. LeetCode - The World's Leading Online Programming Learning Platform. https:\/\/leetcode.com\/."},{"key":"e_1_3_2_1_37_1","volume-title":"Thirty-Second AAAI Conference on Artificial Intelligence","author":"Liang Y.","year":"2018","unstructured":"Liang , Y. , and Zhu , K. Q . Automatic generation of text descriptive comments for code blocks . In Thirty-Second AAAI Conference on Artificial Intelligence ( 2018 ). Liang, Y., and Zhu, K. Q. Automatic generation of text descriptive comments for code blocks. In Thirty-Second AAAI Conference on Artificial Intelligence (2018)."},{"key":"e_1_3_2_1_38_1","first-page":"74","volume-title":"Text Summarization Branches Out (Barcelona","author":"Lin C.-Y.","year":"2004","unstructured":"Lin , C.-Y. ROUGE: A package for automatic evaluation of summaries . In Text Summarization Branches Out (Barcelona , Spain, July 2004 ), Association for Computational Linguistics , pp. 74 -- 81 . Lin, C.-Y. ROUGE: A package for automatic evaluation of summaries. In Text Summarization Branches Out (Barcelona, Spain, July 2004), Association for Computational Linguistics, pp. 74--81."},{"key":"e_1_3_2_1_39_1","doi-asserted-by":"publisher","DOI":"10.1145\/3361242.3362774"},{"key":"e_1_3_2_1_40_1","volume-title":"A neural architecture for generating natural language descriptions from source code changes. arXiv preprint arXiv:1704.04856","author":"Loyola P.","year":"2017","unstructured":"Loyola , P. , Marrese-Taylor , E. , and Matsuo , Y . A neural architecture for generating natural language descriptions from source code changes. arXiv preprint arXiv:1704.04856 ( 2017 ). Loyola, P., Marrese-Taylor, E., and Matsuo, Y. A neural architecture for generating natural language descriptions from source code changes. arXiv preprint arXiv:1704.04856 (2017)."},{"key":"e_1_3_2_1_41_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICICCT.2018.8473199"},{"key":"e_1_3_2_1_42_1","first-page":"279","volume-title":"Proceedings of the 22nd International Conference on Program Comprehension (New York, NY, USA, 2014), ICPC 2014, Association for Computing Machinery","author":"McBurney P. W.","unstructured":"McBurney , P. W. , and McMillan , C. Automatic documentation generation via source code summarization of method context . In Proceedings of the 22nd International Conference on Program Comprehension (New York, NY, USA, 2014), ICPC 2014, Association for Computing Machinery , p. 279 -- 290 . McBurney, P. W., and McMillan, C. Automatic documentation generation via source code summarization of method context. In Proceedings of the 22nd International Conference on Program Comprehension (New York, NY, USA, 2014), ICPC 2014, Association for Computing Machinery, p. 279--290."},{"key":"e_1_3_2_1_43_1","doi-asserted-by":"publisher","DOI":"10.1145\/2597008.2597149"},{"key":"e_1_3_2_1_44_1","first-page":"2","article-title":"Automatic source code summarization of context for java methods","volume":"42","author":"McBurney P. W.","year":"2015","unstructured":"McBurney , P. W. , and McMillan , C . Automatic source code summarization of context for java methods . IEEE Transactions on Software Engineering 42 , 2 ( 2015 ), 103--119. McBurney, P. W., and McMillan, C. Automatic source code summarization of context for java methods. IEEE Transactions on Software Engineering 42, 2 (2015), 103--119.","journal-title":"IEEE Transactions on Software Engineering"},{"key":"e_1_3_2_1_45_1","doi-asserted-by":"publisher","DOI":"10.5220\/0007678100150026"},{"key":"e_1_3_2_1_46_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICPC.2013.6613830"},{"key":"e_1_3_2_1_47_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11390-016-1671-1"},{"key":"e_1_3_2_1_48_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11390-016-1671-1"},{"key":"e_1_3_2_1_49_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11704-015-4409-2"},{"key":"e_1_3_2_1_50_1","doi-asserted-by":"publisher","DOI":"10.5555\/1038267.1039039"},{"key":"e_1_3_2_1_51_1","volume-title":"Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics","author":"Papineni K.","year":"2002","unstructured":"Papineni , K. , Roukos , S. , Ward , T. , and Zhu , W . -J. Bleu: a method for automatic evaluation of machine translation . In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics ( Philadelphia, Pennsylvania, USA , July 2002 ), Association for Computational Linguistics, pp. 311--318. Papineni, K., Roukos, S., Ward, T., and Zhu, W.-J. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics (Philadelphia, Pennsylvania, USA, July 2002), Association for Computational Linguistics, pp. 311--318."},{"key":"e_1_3_2_1_52_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2002.1010061"},{"key":"e_1_3_2_1_53_1","doi-asserted-by":"publisher","DOI":"10.1162\/coli_a_00322"},{"key":"e_1_3_2_1_54_1","doi-asserted-by":"publisher","DOI":"10.1145\/2568225.2568247"},{"key":"e_1_3_2_1_55_1","doi-asserted-by":"publisher","DOI":"10.5555\/2337223.2337254"},{"key":"e_1_3_2_1_56_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICPC.2017.9"},{"key":"e_1_3_2_1_57_1","doi-asserted-by":"publisher","DOI":"10.1145\/1218563.1218587"},{"key":"e_1_3_2_1_58_1","volume-title":"Automatic source code summarization with extended tree-lstm. ArXiv abs\/1906.08094","author":"Shido Y.","year":"2019","unstructured":"Shido , Y. , Kobayashi , Y. , Yamamoto , A. , Miyamoto , A. , and Matsumura , T . Automatic source code summarization with extended tree-lstm. ArXiv abs\/1906.08094 ( 2019 ). Shido, Y., Kobayashi, Y., Yamamoto, A., Miyamoto, A., and Matsumura, T. Automatic source code summarization with extended tree-lstm. ArXiv abs\/1906.08094 (2019)."},{"key":"e_1_3_2_1_59_1","doi-asserted-by":"publisher","DOI":"10.1109\/SANER.2016.35"},{"key":"e_1_3_2_1_60_1","doi-asserted-by":"publisher","DOI":"10.1145\/3106237.3106268"},{"key":"e_1_3_2_1_61_1","doi-asserted-by":"publisher","DOI":"10.1145\/1181775.1181779"},{"key":"e_1_3_2_1_62_1","doi-asserted-by":"publisher","DOI":"10.1145\/1858996.1859006"},{"key":"e_1_3_2_1_63_1","doi-asserted-by":"publisher","DOI":"10.1145\/1985793.1985808"},{"key":"e_1_3_2_1_64_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICPC.2011.28"},{"key":"e_1_3_2_1_65_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11219-006-9216-4"},{"key":"e_1_3_2_1_66_1","doi-asserted-by":"publisher","DOI":"10.1109\/32.6171"},{"key":"e_1_3_2_1_67_1","doi-asserted-by":"publisher","DOI":"10.1109\/2.402076"},{"key":"e_1_3_2_1_68_1","first-page":"3","article-title":"Jsea: A program comprehension tool adopting lda-based topic modeling","volume":"2","author":"Wang T.","year":"2017","unstructured":"Wang , T. , and Liu , Y . Jsea: A program comprehension tool adopting lda-based topic modeling . International Journal of Advanced Computer Science and Applications 2 , 3 ( 2017 ). Wang, T., and Liu, Y. Jsea: A program comprehension tool adopting lda-based topic modeling. International Journal of Advanced Computer Science and Applications 2, 3 (2017).","journal-title":"International Journal of Advanced Computer Science and Applications"},{"key":"e_1_3_2_1_69_1","doi-asserted-by":"publisher","DOI":"10.1145\/1985793.1985868"},{"key":"e_1_3_2_1_70_1","doi-asserted-by":"publisher","DOI":"10.5555\/800078.802534"},{"key":"e_1_3_2_1_71_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2017.2734091"},{"key":"e_1_3_2_1_72_1","volume-title":"Journal of Physics: Conference Series (2019)","author":"Zhao L.","year":"2010","unstructured":"Zhao , L. , Zhang , L. , and Yan , S . A survey on research of code comment auto generation . In Journal of Physics: Conference Series (2019) , vol. 1345 , IOP Publishing , p. 03 2010 . Zhao, L., Zhang, L., and Yan, S. A survey on research of code comment auto generation. In Journal of Physics: Conference Series (2019), vol. 1345, IOP Publishing, p. 032010."}],"event":{"name":"ICPC '20: 28th International Conference on Program Comprehension","location":"Seoul Republic of Korea","acronym":"ICPC '20","sponsor":["SIGSOFT ACM Special Interest Group on Software Engineering","IEEE CS"]},"container-title":["Proceedings of the 28th International Conference on Program Comprehension"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3387904.3389258","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3387904.3389258","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T22:41:37Z","timestamp":1750200097000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3387904.3389258"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,7,13]]},"references-count":72,"alternative-id":["10.1145\/3387904.3389258","10.1145\/3387904"],"URL":"https:\/\/doi.org\/10.1145\/3387904.3389258","relation":{},"subject":[],"published":{"date-parts":[[2020,7,13]]},"assertion":[{"value":"2020-09-12","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}