{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,7]],"date-time":"2026-06-07T00:14:37Z","timestamp":1780791277052,"version":"3.54.1"},"reference-count":57,"publisher":"Springer Science and Business Media LLC","issue":"4","license":[{"start":{"date-parts":[[2023,5,23]],"date-time":"2023-05-23T00:00:00Z","timestamp":1684800000000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2023,5,23]],"date-time":"2023-05-23T00:00:00Z","timestamp":1684800000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100006447","name":"University of Zurich","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100006447","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Empir Software Eng"],"published-print":{"date-parts":[[2023,7]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Source code comments are a cornerstone of software documentation facilitating feature development and maintenance. Well-defined documentation formats, like Javadoc, make it easy to include structural metadata used to, for example, generate documentation manuals. However, the actual usage of structural elements in source code comments has not been studied yet. We investigate to which extent these structural elements are used in practice and whether the added information can be leveraged to improve tools assisting developers when writing comments. Existing research on comment generation traditionally focuses on automatic generation of summaries. However, recent works have shown promising results when supporting comment authoring through a <jats:italic>next-word<\/jats:italic> prediction. In this paper, we present an in-depth analysis of commenting practice in more than 18K open-source projects written in Python and Java showing that many structural elements, particularly parameter and return value descriptions are indeed widely used. We discover that while a majority are rather short at about 6 to 9 words, many are several hundred words in length. We further find that Python comments tend to be significantly longer than Java comments, possibly due to the weakly-typed nature of the former. Following the empirical analysis, we extend an existing language model with support for structural information, substantially improving the Top-1 accuracy of predicted words (Python 9.6%, Java 7.8%).<\/jats:p>","DOI":"10.1007\/s10664-022-10284-6","type":"journal-article","created":{"date-parts":[[2023,5,23]],"date-time":"2023-05-23T10:02:14Z","timestamp":1684836134000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":6,"title":["Completing Function Documentation Comments Using Structural Information"],"prefix":"10.1007","volume":"28","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-8874-5450","authenticated-orcid":false,"given":"Adelina","family":"Ciurumelea","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Carol V.","family":"Alexandru","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Harald C.","family":"Gall","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sebastian","family":"Proksch","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2023,5,23]]},"reference":[{"key":"10284_CR1","doi-asserted-by":"publisher","unstructured":"Aghajani E, Nagy C, Linares-V\u00e1squez M, Moreno L, Bavota G, Lanza M, Shepherd DC (2020) Software documentation: the practitioners\u2019 perspective. In: Proceedings of the ACM\/IEEE 42nd international conference on software engineering, ICSE \u201920, Association for computing machinery, New York, pp 590\u2013601. https:\/\/doi.org\/10.1145\/3377811.3380405","DOI":"10.1145\/3377811.3380405"},{"key":"10284_CR2","doi-asserted-by":"publisher","unstructured":"Aghajani E, Nagy C, Vega-M\u00e1rquez OL, Linares-V\u00e1squez M, Moreno L, Bavota G, Lanza M (2019) Software documentation issues unveiled. In: Proceedings of the 41st international conference on software engineering, ICSE \u201919, IEEE Press, pp 1199\u20131210. https:\/\/doi.org\/10.1109\/ICSE.2019.00122","DOI":"10.1109\/ICSE.2019.00122"},{"key":"10284_CR3","doi-asserted-by":"crossref","unstructured":"Allamanis M (2018) The adverse effects of code duplication in machine learning models of code. CoRR arXiv:1812.06469","DOI":"10.1145\/3359591.3359735"},{"key":"10284_CR4","unstructured":"Alon U, Levy O, Yahav E (2018) code2seq: generating sequences from structured representations of code. CoRR arXiv:1808.01400"},{"key":"10284_CR5","doi-asserted-by":"publisher","unstructured":"Bruch M, Monperrus M, Mezini M (2009) Learning from examples to improve code completion systems. In: Proceedings of the 7th joint meeting of the european software engineering conference and the ACM sigsoft symposium on the foundations of software engineering, ESEC\/FSE \u201909, Association for computing machinery, New York, pp 213\u2013222. https:\/\/doi.org\/10.1145\/1595696.1595728","DOI":"10.1145\/1595696.1595728"},{"key":"10284_CR6","doi-asserted-by":"publisher","unstructured":"Chen MX, Lee BN, Bansal G, Cao Y, Zhang S, Lu J, Tsay J, Wang Y, Dai AM, Chen Z, Sohn T, Wu Y (2019) Gmail smart compose: Real-time assisted writing. In: Proceedings of the 25th ACM sigkdd international conference on knowledge discovery & data mining, KDD \u201919, association for computing machinery, New York, pp 2287\u20132295.. https:\/\/doi.org\/10.1145\/3292500.3330723","DOI":"10.1145\/3292500.3330723"},{"key":"10284_CR7","doi-asserted-by":"publisher","unstructured":"Chen Q, Xia X, Hu H, Lo D, Li S (2021) Why my code summarization model does not work: Code comment improvement with category prediction. ACM Trans Softw Eng Methodol, vol 30(2). https:\/\/doi.org\/10.1145\/3434280","DOI":"10.1145\/3434280"},{"key":"10284_CR8","doi-asserted-by":"publisher","unstructured":"Ciurumelea A, Proksch S, Gall HC (2020) Suggesting comment completions for python using neural language models. In: 2020 IEEE 27th international conference on software analysis, evolution and reengineering (SANER), pp 456\u2013467. https:\/\/doi.org\/10.1109\/SANER48275.2020.9054866","DOI":"10.1109\/SANER48275.2020.9054866"},{"key":"10284_CR9","unstructured":"Docstring parser (2021) https:\/\/pypi.org\/project\/docstring-parser\/. Accessed 01 June 2021"},{"key":"10284_CR10","unstructured":"Doctest \u2014 test interactive python examples (2021) https:\/\/docs.python.org\/3\/library\/doctest.html. Accessed 12 Apr 2021"},{"key":"10284_CR11","unstructured":"Example numpy style python docstrings (2022) https:\/\/sphinxcontrib-napoleon.readthedocs.io\/en\/latest\/example_numpy.html#example-numpy. Accessed 16 Mar 2022"},{"key":"10284_CR12","doi-asserted-by":"publisher","unstructured":"Fluri B, Wursch M, Gall HC (2007) Do code and comments co-evolve? on the relation between source code and comment changes. In: 14th Working conference on reverse engineering (WCRE 2007), pp 70\u201379. https:\/\/doi.org\/10.1109\/WCRE.2007.21","DOI":"10.1109\/WCRE.2007.21"},{"key":"10284_CR13","unstructured":"Github rest api (2022) https:\/\/docs.github.com\/en\/rest. Accessed 16 Mar 2022"},{"key":"10284_CR14","unstructured":"Google cloud public datasets (2021) https:\/\/cloud.google.com\/public-datasets. Accessed 01 June 2021"},{"key":"10284_CR15","unstructured":"Google docstring style (2022) https:\/\/github.com\/google\/styleguide\/blob\/gh-pages\/pyguide.md#38-comments-and-docstrings. Accessed 15 Mar 2022"},{"key":"10284_CR16","doi-asserted-by":"publisher","unstructured":"Haiduc S, Aponte J, Moreno L, Marcus A (2010) On the use of automated text summarization techniques for summarizing source code. In: 2010 17th Working conference on reverse engineering, pp 35\u201344. https:\/\/doi.org\/10.1109\/WCRE.2010.13","DOI":"10.1109\/WCRE.2010.13"},{"key":"10284_CR17","doi-asserted-by":"publisher","unstructured":"Haouari D, Sahraoui H, Langlais P (2011) How good is your comment? a study of comments in java programs. In: 2011 International symposium on empirical software engineering and measurement, pp 137\u2013146 https:\/\/doi.org\/10.1109\/ESEM.2011.22","DOI":"10.1109\/ESEM.2011.22"},{"key":"10284_CR18","doi-asserted-by":"crossref","unstructured":"Haque S, Le Clair A, Wu L, McMillan C (2020) Improved automatic summarization of subroutines via attention to file context. CoRR arXiv:2004.04881","DOI":"10.1145\/3379597.3387449"},{"key":"10284_CR19","doi-asserted-by":"publisher","unstructured":"Hellendoorn VJ, Devanbu P (2017) Are deep neural networks the best choice for modeling source code?. In: Proceedings of the 2017 11th joint meeting on foundations of software engineering, ESEC\/FSE 2017, ACM, New York, pp 763\u2013773. https:\/\/doi.org\/10.1145\/3106237.3106290","DOI":"10.1145\/3106237.3106290"},{"key":"10284_CR20","unstructured":"How to write doc comments for the javadoc tool (2021) https:\/\/www.oracle.com\/technical-resources\/articles\/java\/javadoc-tool.html. Accessed: 22 Mar 2021"},{"key":"10284_CR21","doi-asserted-by":"crossref","unstructured":"Hu X, Li G, Xia X, Lo D, Jin Z (2018) Deep code comment generation. In: Proceedings of the 26th conference on program comprehension, ICPC \u201918, association for computing machinery, New York pp 200\u2013210. https:\/\/doi.org\/doi.org\/10.1145\/3196321.3196334","DOI":"10.1145\/3196321.3196334"},{"key":"10284_CR22","doi-asserted-by":"publisher","unstructured":"Hu X, Li G, Xia X, Lo D, Jin Z (2020) Deep code comment generation with hybrid lexical and syntactical information. Empir Softw Eng, vol 25. https:\/\/doi.org\/10.1145\/3196321.3196334","DOI":"10.1145\/3196321.3196334"},{"key":"10284_CR23","unstructured":"Husain H, Wu H, Gazit T, Allamanis M, Brockschmidt M (2019) Codesearchnet challenge: Evaluating the state of semantic code search. CoRR arXiv:1909.09436"},{"key":"10284_CR24","unstructured":"Husain H, Wu HH, Gazit T, Allamanis M, Brockschmidt M (2019) CodeSearchNet, challenge: evaluating the state of semantic code search. arXiv:1909.09436"},{"key":"10284_CR25","unstructured":"Intellij idea - code completion (2021) https:\/\/www.jetbrains.com\/help\/idea\/auto-completing-code.html. Accessed 01 June 2021"},{"key":"10284_CR26","unstructured":"Javalang (2021) https:\/\/github.com\/c2nes\/javalang. Accessed 29 Mar 2021"},{"key":"10284_CR27","doi-asserted-by":"publisher","unstructured":"Iyer S, Konstas I, Cheung A, Zettlemoyer L (2016) Summarizing source code using a neural attention model. In: Proceedings of the 54th annual meeting of the association for computational linguistics, Association for computational linguistics, Berlin (vol 1: long papers), pp 2073\u20132083. https:\/\/doi.org\/10.18653\/v1\/P16-1195. https:\/\/www.aclweb.org\/anthology\/P16-1195","DOI":"10.18653\/v1\/P16-1195"},{"key":"10284_CR28","unstructured":"Jurafsky D, Martin JH (2000) Speech and language processing: an introduction to natural language processing, computational linguistics, and speech recognition, 1st edn. Prentice Hall PTR, Upper Saddle River"},{"issue":"2-3","key":"10284_CR29","doi-asserted-by":"publisher","first-page":"259","DOI":"10.1080\/01638539809545028","volume":"25","author":"TK Landauer","year":"1998","unstructured":"Landauer TK, Foltz PW, Laham D (1998) An introduction to latent semantic analysis. Discourse Process 25(2-3):259\u2013284. https:\/\/doi.org\/10.1080\/01638539809545028","journal-title":"Discourse Process"},{"key":"10284_CR30","unstructured":"Langdetect (2021) https:\/\/pypi.org\/project\/langdetect\/. Accessed: 01 June 2021"},{"key":"10284_CR31","doi-asserted-by":"publisher","unstructured":"Le Clair A, Haque S, Wu L, McMillan C (2020) Improved code summarization via a graph neural network. In: Proceedings of the 28th international conference on program comprehension, ICPC \u201920, association for computing machinery, New York, NY pp 184\u2013195. https:\/\/doi.org\/10.1145\/3387904.3389268","DOI":"10.1145\/3387904.3389268"},{"key":"10284_CR32","doi-asserted-by":"publisher","unstructured":"Le Clair A, Jiang S, McMillan C, A neural model for generating natural language summaries of program subroutines (2019). In: Proceedings of the 41st international conference on software engineering, ICSE \u201919. IEEE Press pp 795\u2013806. https:\/\/doi.org\/10.1109\/ICSE.2019.00087","DOI":"10.1109\/ICSE.2019.00087"},{"key":"10284_CR33","doi-asserted-by":"crossref","unstructured":"Le Clair A, McMillan C (2019) Recommendations for datasets for source code summarization. CoRR arXiv:1904.02660","DOI":"10.18653\/v1\/N19-1394"},{"key":"10284_CR34","unstructured":"Lib2to3 (2022) https:\/\/github.com\/python\/cpython\/tree\/3.10\/Lib\/lib2to3\/. Accessed 12 Mar 2022"},{"key":"10284_CR35","unstructured":"Lin CY (2004) ROUGE: a package for automatic evaluation of summaries. In: Text summarization branches out, association for computational linguistics, Barcelona, pp 74\u201381. https:\/\/aclanthology.org\/W04-1013"},{"key":"10284_CR36","doi-asserted-by":"crossref","unstructured":"Mastropaolo A, Scalabrino S, Cooper N, Nader-Palacio D, Poshyvanyk D, Oliveto R, Bavota G (2021) Studying the usage of text-to-text transfer transformer to support code-related tasks. CoRR arXiv:2102.02017","DOI":"10.1109\/ICSE43902.2021.00041"},{"key":"10284_CR37","doi-asserted-by":"publisher","unstructured":"Moreno L, Aponte J, Sridhara G, Marcus A, Pollock L, Vijay-Shanker K (2013) Automatic generation of natural language summaries for java classes. In: 2013 21st International conference on program comprehension (ICPC), pp 23\u201332. https:\/\/doi.org\/10.1109\/ICPC.2013.6613830","DOI":"10.1109\/ICPC.2013.6613830"},{"key":"10284_CR38","unstructured":"Nltk sentence tokenizer (2021) https:\/\/www.nltk.org\/api\/nltk.tokenize.html#module-nltk.tokenize.punkt. Accessed 01 June 2021"},{"key":"10284_CR39","unstructured":"Numpy docstring style (2022) https:\/\/numpydoc.readthedocs.io\/en\/latest\/format.html#docstring-standard. Accessed 15 Mar 2022"},{"key":"10284_CR40","unstructured":"Numpydoc docstring guide (2021) https:\/\/numpydoc.readthedocs.io\/en\/latest\/format.html#method-docstrings. Accessed 01 June 2021"},{"key":"10284_CR41","doi-asserted-by":"publisher","unstructured":"Pascarella L, Bacchelli A (2017) Classifying code comments in java open-source software systems. In: Proceedings of the 14th international conference on mining software repositories, MSR \u201917. IEEE press, Piscataway, pp 227\u2013237. https:\/\/doi.org\/10.1109\/MSR.2017.63","DOI":"10.1109\/MSR.2017.63"},{"key":"10284_CR42","unstructured":"Pep 257 \u2013 docstring conventions (2021) https:\/\/www.python.org\/dev\/peps\/pep-0257\/. Accessed 01 June 2021"},{"key":"10284_CR43","unstructured":"Pep 287 \u2013 restructuredtext docstring format (2021) https:\/\/www.python.org\/dev\/peps\/pep-0287\/. Accessed 01 June 2021"},{"key":"10284_CR44","unstructured":"Python rouge implementation (2022) https:\/\/pypi.org\/project\/rouge-score\/. Accessed 09 Sept 2022"},{"key":"10284_CR45","unstructured":"Raffel C, Shazeer N, Roberts A, Lee K, Narang S, Matena M, Zhou Y, Li W, Liu PJ (2019) Exploring the limits of transfer learning with a unified text-to-text transformer. CoRR arXiv:1910.10683"},{"key":"10284_CR46","unstructured":"Replication package (2022) - completing function documentation comments using structural information. https:\/\/www.dropbox.com\/sh\/vi2zl3shvntp8p7\/AAC5uwwDV973h2L1Ou7A4hcYa?dl=0. Accessed 16 Dec 2022"},{"issue":"11","key":"10284_CR47","doi-asserted-by":"publisher","first-page":"613","DOI":"10.1145\/361219.361220","volume":"18","author":"G Salton","year":"1975","unstructured":"Salton G, Wong A, Yang CS (1975) A vector space model for automatic indexing. Commun ACM 18(11):613\u2013620. https:\/\/doi.org\/10.1145\/361219.361220","journal-title":"Commun ACM"},{"key":"10284_CR48","doi-asserted-by":"publisher","unstructured":"Sridhara G, Hill E, Pollock L, Vijay-Shanker K (2010) Towards automatically generating summary comments for java methods. In: Proceedings of the IEEE\/ACM international conference on automated software engineering, ASE \u201910, ACM, New York, pp 43\u201352. https:\/\/doi.org\/10.1145\/1858996.1859006","DOI":"10.1145\/1858996.1859006"},{"key":"10284_CR49","doi-asserted-by":"publisher","unstructured":"Steidl D, Hummel B, Juergens E (2013) Quality analysis of source code comments. In: 2013 21st International conference on program comprehension (ICPC), pp 83\u201392. https:\/\/doi.org\/10.1109\/ICPC.2013.6613836","DOI":"10.1109\/ICPC.2013.6613836"},{"key":"10284_CR50","doi-asserted-by":"crossref","unstructured":"Svyatkovskiy A, Deng SK, Fu S, Sundaresan N (2020) Intellicode compose: Code generation using transformer. CoRR arXiv:2005.08025","DOI":"10.1145\/3368089.3417058"},{"key":"10284_CR51","unstructured":"The epytext markup language (2022) http:\/\/epydoc.sourceforge.net\/manual-epytext.html. Accessed 15 Mar 2022"},{"key":"10284_CR52","doi-asserted-by":"publisher","unstructured":"Wan Y, Zhao Z, Yang M, Xu G, Ying H, Wu J, Yu PS (2018) Improving automatic source code summarization via deep reinforcement learning. In: Proceedings of the 33rd ACM\/IEEE international conference on automated software engineering, ASE 2018, association for computing machinery, New York, pp 397\u2013407. https:\/\/doi.org\/10.1145\/3238147.3238206","DOI":"10.1145\/3238147.3238206"},{"key":"10284_CR53","doi-asserted-by":"publisher","unstructured":"Wen F, Nagy C, Bavota G, Lanza M (2019) A large-scale empirical study on code-comment inconsistencies. In: 2019 IEEE\/ACM 27th International conference on program comprehension (ICPC), pp 53\u201364. https:\/\/doi.org\/10.1109\/ICPC.2019.00019","DOI":"10.1109\/ICPC.2019.00019"},{"key":"10284_CR54","unstructured":"Writing system software: code comments (2021) http:\/\/antirez.com\/news\/124. Accessed 01 June 2021"},{"issue":"10","key":"10284_CR55","doi-asserted-by":"publisher","first-page":"951","DOI":"10.1109\/TSE.2017.2734091","volume":"44","author":"X Xia","year":"2018","unstructured":"Xia X, Bao L, Lo D, Xing Z, Hassan AE, Li S (2018) Measuring program comprehension: a large-scale field study with professionals. IEEE Trans Softw Eng 44(10):951\u2013976. https:\/\/doi.org\/10.1109\/TSE.2017.2734091","journal-title":"IEEE Trans Softw Eng"},{"key":"10284_CR56","doi-asserted-by":"publisher","unstructured":"Ye D, Xing Z, Li J, Kapre N (2016) Software-specific part-of-speech tagging: An experimental study on stack overflow. In: Proceedings of the 31st annual ACM symposium on applied computing, SAC \u201916, association for computing machinery, New York, pp 1378\u20131385. https:\/\/doi.org\/10.1145\/2851613.2851772","DOI":"10.1145\/2851613.2851772"},{"key":"10284_CR57","doi-asserted-by":"publisher","unstructured":"Zhou W, Kim S, Murali V, Aye GA (2022) Improving code autocompletion with transfer learning. In: 2022 IEEE\/ACM 44th international conference on software engineering: Software engineering in practice (ICSE-SEIP), pp 161\u2013162. https:\/\/doi.org\/10.1145\/3510457.3513061","DOI":"10.1145\/3510457.3513061"}],"container-title":["Empirical Software Engineering"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10664-022-10284-6.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10664-022-10284-6\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10664-022-10284-6.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,9,27]],"date-time":"2023-09-27T09:14:26Z","timestamp":1695806066000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10664-022-10284-6"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,5,23]]},"references-count":57,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2023,7]]}},"alternative-id":["10284"],"URL":"https:\/\/doi.org\/10.1007\/s10664-022-10284-6","relation":{},"ISSN":["1382-3256","1573-7616"],"issn-type":[{"value":"1382-3256","type":"print"},{"value":"1573-7616","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,5,23]]},"assertion":[{"value":"12 December 2022","order":1,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"23 May 2023","order":2,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors declare that they have no competing interests.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"<!--Emphasis Type='Bold' removed-->Competing interests"}}],"article-number":"86"}}