{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,23]],"date-time":"2026-10-23T20:30:36Z","timestamp":1792787436257,"version":"4.5.0"},"publisher-location":"New York, NY, USA","reference-count":53,"publisher":"ACM","license":[{"start":{"date-parts":[[2024,4,20]],"date-time":"2024-04-20T00:00:00Z","timestamp":1713571200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/100000001","name":"NSF (National Science Foundation)","doi-asserted-by":"publisher","award":["CCF-1955853"],"award-info":[{"award-number":["CCF-1955853"]}],"id":[{"id":"10.13039\/100000001","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Horizon 2020","award":["957254-COSMOS"],"award-info":[{"award-number":["957254-COSMOS"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2024,4,20]]},"DOI":"10.1145\/3643787.3648038","type":"proceedings-article","created":{"date-parts":[[2024,8,26]],"date-time":"2024-08-26T09:06:02Z","timestamp":1724663162000},"page":"33-40","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":3,"title":["The NLBSE'24 Tool Competition"],"prefix":"10.1145","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-5402-9114","authenticated-orcid":false,"given":"Rafael","family":"Kallis","sequence":"first","affiliation":[{"name":"Rafael Kallis Consulting, Zurich, Switzerland"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3871-401X","authenticated-orcid":false,"given":"Giuseppe","family":"Colavito","sequence":"additional","affiliation":[{"name":"University of Bari, Bari, Italy"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7338-2044","authenticated-orcid":false,"given":"Ali","family":"Al-Kaswan","sequence":"additional","affiliation":[{"name":"Delft University of Technology, Delft, The Netherlands"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8289-7736","authenticated-orcid":false,"given":"Luca","family":"Pascarella","sequence":"additional","affiliation":[{"name":"ETH Zurich, Zurich, Switzerland"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2838-685X","authenticated-orcid":false,"given":"Oscar","family":"Chaparro","sequence":"additional","affiliation":[{"name":"College of William & Mary, Williamsburg, Virginia, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5127-4042","authenticated-orcid":false,"given":"Pooja","family":"Rani","sequence":"additional","affiliation":[{"name":"University of Zurich, Zurich, Switzerland"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2024,8,26]]},"reference":[{"key":"e_1_3_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jss.2020.110800"},{"key":"e_1_3_2_1_2_1","volume-title":"Proceedings of The 2nd International Workshop on Natural Language-based Software Engineering (NLBSE'23)","author":"Al-Kaswan Ali","year":"2023","unstructured":"Ali Al-Kaswan, Maliheh Izadi, and Arie van Deursen. 2023. STACC: Code Comment Classification using Sentence Transformers. In Proceedings of The 2nd International Workshop on Natural Language-based Software Engineering (NLBSE'23). (to appear)."},{"key":"e_1_3_2_1_3_1","volume-title":"Proceedings of The 3rd International Workshop on Natural Language-based Software Engineering (NLBSE'24)","author":"Alam Khubain Amjad","year":"2024","unstructured":"Khubain Amjad Alam, Ashish Jumani, Harris Aamir, and Muhammad Uzair. 2024. ClassifAI: Automating Issue Reports Classification using Pre-Trained BERT (Bidirectional Encoder Representations from Transformers) Models. In Proceedings of The 3rd International Workshop on Natural Language-based Software Engineering (NLBSE'24). Lisbon, Portugal."},{"key":"e_1_3_2_1_4_1","volume-title":"Proceedings of The 3rd International Workshop on Natural Language-based Software Engineering (NLBSE'24)","author":"Aracena Gabriel","year":"2024","unstructured":"Gabriel Aracena, Kyle Luster, Fabio Santos, Igor Steinmacher, and Marco Aurelio Gerosa. 2024. Applying Large Language Models API to Issue Classification Problem. In Proceedings of The 3rd International Workshop on Natural Language-based Software Engineering (NLBSE'24). Lisbon, Portugal."},{"key":"e_1_3_2_1_5_1","volume-title":"Lin (Eds.)","volume":"33","author":"Brown Tom","year":"2020","unstructured":"Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chris Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. 2020. Language Models are Few-Shot Learners. In Advances in Neural Information Processing Systems, H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin (Eds.), Vol. 33. Curran Associates, Inc., 1877--1901. https:\/\/proceedings.neurips.cc\/paper_files\/paper\/2020\/file\/1457c0d6bfcb4967418bfb8ac142f64a-Paper.pdf"},{"key":"e_1_3_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1109\/SANER.2015.7081875"},{"key":"e_1_3_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1145\/3338906.3338947"},{"key":"e_1_3_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1145\/3106237.3106285"},{"key":"e_1_3_2_1_9_1","unstructured":"Hyung Won Chung Le Hou Shayne Longpre Barret Zoph Yi Tay William Fedus Eric Li Xuezhi Wang Mostafa Dehghani Siddhartha Brahma et al. 2022. Scaling instruction-finetuned language models. arXiv preprint arXiv:2210.11416 (2022)."},{"key":"e_1_3_2_1_10_1","volume-title":"Proceedings of The 2nd International Workshop on Natural Language-based Software Engineering (NLBSE'23)","author":"Colavito Giuseppe","year":"2023","unstructured":"Giuseppe Colavito, Filippo Lanubile, and Nicole Novielli. 2023. Few-Shot Learning for Issue Report Classification. In Proceedings of The 2nd International Workshop on Natural Language-based Software Engineering (NLBSE'23). (to appear)."},{"key":"e_1_3_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1145\/1085313.1085331"},{"key":"e_1_3_2_1_12_1","article-title":"Investigating the criticality of user-reported issues through their relations with app rating","volume":"33","author":"Sorbo Andrea Di","year":"2021","unstructured":"Andrea Di Sorbo, Giovanni Grano, Corrado Aaron Visaggio, and Sebastiano Panichella. 2021. Investigating the criticality of user-reported issues through their relations with app rating. J. Softw. Evol. Process. 33, 3 (2021).","journal-title":"J. Softw. Evol. Process."},{"key":"e_1_3_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1145\/3573074.3573101"},{"key":"e_1_3_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1145\/2950290.2950299"},{"key":"e_1_3_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICSME52107.2021.00058"},{"key":"e_1_3_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1145\/3564821"},{"key":"e_1_3_2_1_17_1","volume-title":"Proceedings of The 3rd International Workshop on Natural Language-based Software Engineering (NLBSE'24)","author":"Ebrahim Fahad","year":"2024","unstructured":"Fahad Ebrahim and Mike Joy. 2024. Few-Shot Issue Report Classification with Adapters. In Proceedings of The 3rd International Workshop on Natural Language-based Software Engineering (NLBSE'24). Lisbon, Portugal."},{"key":"e_1_3_2_1_18_1","volume-title":"Codebert: A pre-trained model for programming and natural languages. arXiv preprint arXiv:2002.08155","author":"Feng Zhangyin","year":"2020","unstructured":"Zhangyin Feng, Daya Guo, Duyu Tang, Nan Duan, Xiaocheng Feng, Ming Gong, Linjun Shou, Bing Qin, Ting Liu, Daxin Jiang, and others. 2020. Codebert: A pre-trained model for programming and natural languages. arXiv preprint arXiv:2002.08155 (2020)."},{"key":"e_1_3_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.acl-main.520"},{"key":"e_1_3_2_1_20_1","volume-title":"Proceedings of The 3rd International Workshop on Natural Language-based Software Engineering (NLBSE'24)","author":"G\u00f3mez-Barrera Daniel","year":"2024","unstructured":"Daniel G\u00f3mez-Barrera, Luccas Rojas Becerra, Juan Pinz\u00f3n Roncancio, David Ortiz Almanza, Juan Arboleda, Mario Linares, and Ruben Manrique. 2024. Lessons from the NLBSE 2024 Competition: Towards Building Efficient Models for GitHub Issue Classification. In Proceedings of The 3rd International Workshop on Natural Language-based Software Engineering (NLBSE'24). Lisbon, Portugal."},{"key":"e_1_3_2_1_21_1","volume-title":"Proceedings of The 3rd International Workshop on Natural Language-based Software Engineering (NLBSE'24)","author":"Hai Nam Le","year":"2024","unstructured":"Nam Le Hai and Nghi DQ Bui. 2024. Dopamin: Transformer-based Comment Classifiers through Domain Post-Training and Multi-level Layer Aggregation. In Proceedings of The 3rd International Workshop on Natural Language-based Software Engineering (NLBSE'24). Lisbon, Portugal."},{"key":"e_1_3_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10664-020-09885-w"},{"key":"e_1_3_2_1_23_1","first-page":"2640","volume-title":"Proceedings of the 36th International Conference on Machine Learning. PMLR, 2790--2799","author":"Houlsby Neil","year":"2019","unstructured":"Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin De Laroussilhe, Andrea Gesmundo, Mona Attariyan, and Sylvain Gelly. 2019. Parameter-Efficient Transfer Learning for NLP. In Proceedings of the 36th International Conference on Machine Learning. PMLR, 2790--2799. https:\/\/proceedings.mlr.press\/v97\/houlsby19a.html ISSN: 2640-3498."},{"key":"e_1_3_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1145\/3528588.3528662"},{"key":"e_1_3_2_1_25_1","volume-title":"2022 IEEE\/ACM 1st International Workshop on Natural Language-Based Software Engineering (NLBSE). IEEE, 48--50","author":"Izadi Maliheh","year":"2022","unstructured":"Maliheh Izadi and Matin Nili Ahmadabadi. 2022. On the evaluation of NLP-based models for software engineering. In 2022 IEEE\/ACM 1st International Workshop on Natural Language-Based Software Engineering (NLBSE). IEEE, 48--50."},{"key":"e_1_3_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10664-021-10085-3"},{"key":"e_1_3_2_1_27_1","doi-asserted-by":"crossref","unstructured":"Armand Joulin Edouard Grave Piotr Bojanowski and Tomas Mikolov. 2016. Bag of Tricks for Efficient Text Classification. http:\/\/arxiv.org\/abs\/1607.01759 arXiv:1607.01759 [cs].","DOI":"10.18653\/v1\/E17-2068"},{"key":"e_1_3_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1145\/3528588.3528664"},{"key":"e_1_3_2_1_29_1","volume-title":"Proceedings of The 3rd International Workshop on Natural Language-based Software Engineering (NLBSE'24)","author":"Kallis Rafael","year":"2024","unstructured":"Rafael Kallis, Giuseppe Colavito, Oscar Chaparro, Luca Pascarella, Ali Al-Kaswan, and Pooja Rani. 2024. The NLBSE'24 Tool Competition. In Proceedings of The 3rd International Workshop on Natural Language-based Software Engineering (NLBSE'24)."},{"key":"e_1_3_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICSME.2019.00070"},{"key":"e_1_3_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.scico.2020.102598"},{"key":"e_1_3_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1109\/NLBSE59153.2023.00007"},{"key":"e_1_3_2_1_33_1","volume-title":"Proceedings of The 2nd International Workshop on Natural Language-based Software Engineering (NLBSE'23)","author":"Laiq Muhammad","year":"2023","unstructured":"Muhammad Laiq. 2023. An Intelligent Tool for Classifying Issue Reports. In Proceedings of The 2nd International Workshop on Natural Language-based Software Engineering (NLBSE'23). (to appear)."},{"key":"e_1_3_2_1_34_1","volume-title":"ALBERT: A Lite BERT for Self-supervised Learning of Language Representations. In International Conference on Learning Representations (ICLR). https:\/\/openreview.net\/forum?id=H1eA7AEtvS","author":"Lan Zhenzhong","year":"2020","unstructured":"Zhenzhong Lan, Mingda Chen, Sebastian Goodman, Kevin Gimpel, Piyush Sharma, and Radu Soricut. 2020. ALBERT: A Lite BERT for Self-supervised Learning of Language Representations. In International Conference on Learning Representations (ICLR). https:\/\/openreview.net\/forum?id=H1eA7AEtvS"},{"key":"e_1_3_2_1_35_1","doi-asserted-by":"publisher","unstructured":"Yinhan Liu Myle Ott Naman Goyal Jingfei Du Mandar Joshi Danqi Chen Omer Levy Mike Lewis Luke Zettlemoyer and Veselin Stoyanov. 2019. RoBERTa: A Robustly Optimized BERT Pretraining Approach. arXiv:1907.11692 [cs]. 10.48550\/arXiv.1907.11692","DOI":"10.48550\/arXiv.1907.11692"},{"key":"e_1_3_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.1109\/VST.2018.8327148"},{"key":"e_1_3_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.infsof.2021.106665"},{"key":"e_1_3_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.1145\/3617946.3617957"},{"key":"e_1_3_2_1_39_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICSM.2015.7332474"},{"key":"e_1_3_2_1_40_1","doi-asserted-by":"publisher","DOI":"10.1109\/MSR.2017.63"},{"key":"e_1_3_2_1_41_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.emnlp-demos.7"},{"key":"e_1_3_2_1_42_1","volume-title":"Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing: System Demonstrations. Association for Computational Linguistics","author":"Poth Clifton","year":"2023","unstructured":"Clifton Poth, Hannah Sterz, Indraneil Paul, Sukannya Purkayastha, Leon Engl\u00e4nder, Timo Imhof, Ivan Vuli\u0107, Sebastian Ruder, Iryna Gurevych, and Jonas Pfeiffer. 2023. Adapters: A Unified Library for Parameter-Efficient and Modular Transfer Learning. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing: System Demonstrations. Association for Computational Linguistics, Singapore, 149--160. https:\/\/aclanthology.org\/2023.emnlp-demo.13"},{"key":"e_1_3_2_1_43_1","article-title":"Exploring the limits of transfer learning with a unified text-to-text transformer","volume":"21","author":"Raffel Colin","year":"2020","unstructured":"Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2020. Exploring the limits of transfer learning with a unified text-to-text transformer. J. Mach. Learn. Res. 21, 1, Article 140 (jan 2020), 67 pages.","journal-title":"J. Mach. Learn. Res."},{"key":"e_1_3_2_1_44_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jss.2022.111515"},{"key":"e_1_3_2_1_45_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jss.2021.111047"},{"key":"e_1_3_2_1_46_1","doi-asserted-by":"publisher","unstructured":"Nils Reimers and Iryna Gurevych. 2019. Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks. arXiv:1908.10084 [cs]. 10.48550\/arXiv.1908.10084","DOI":"10.48550\/arXiv.1908.10084"},{"key":"e_1_3_2_1_47_1","volume-title":"Proceedings of The 3rd International Workshop on Natural Language-based Software Engineering (NLBSE'24)","author":"Rejithkumar Gokul","year":"2024","unstructured":"Gokul Rejithkumar, Preethu Rose Anish, and Smita Ghaisas. 2024. Text-To-Text Generation for Issue Report Classification. In Proceedings of The 3rd International Workshop on Natural Language-based Software Engineering (NLBSE'24). Lisbon, Portugal."},{"key":"e_1_3_2_1_48_1","doi-asserted-by":"publisher","DOI":"10.1145\/3368089.3417928"},{"key":"e_1_3_2_1_49_1","volume-title":"Proceedings of the 45th International Conference on Software Engineering (ICSE). (to appear).","author":"Song Yang","year":"2023","unstructured":"Yang Song, Junayed Mahmud, Nadeeshan De Silva, Ying Zhou, Oscar Chaparro, Kevin Moran, Andrian Marcus, and Denys Poshyvanyk. 2023. BURT: A Chatbot for Interactive Bug Reporting. In Proceedings of the 45th International Conference on Software Engineering (ICSE). (to appear)."},{"key":"e_1_3_2_1_50_1","doi-asserted-by":"publisher","DOI":"10.1145\/3540250.3549131"},{"key":"e_1_3_2_1_51_1","volume-title":"Hashimoto","author":"Taori Rohan","year":"2023","unstructured":"Rohan Taori, Ishaan Gulrajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto. 2023. Stanford Alpaca: An Instruction-following LLaMA model. https:\/\/github.com\/tatsu-lab\/stanford_alpaca."},{"key":"e_1_3_2_1_52_1","volume-title":"Luke Bates, Daniel Korat, Moshe Wasserblat, and Oren Pereg.","author":"Tunstall Lewis","year":"2022","unstructured":"Lewis Tunstall, Nils Reimers, Unso Eun Seo Jo, Luke Bates, Daniel Korat, Moshe Wasserblat, and Oren Pereg. 2022. Efficient Few-Shot Learning Without Prompts. arXiv preprint arXiv:2209.11055 (2022)."},{"key":"e_1_3_2_1_53_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10462-004-0751-8"}],"event":{"name":"NLBSE '24: Third ACM\/IEEE International Workshop on NL-based Software Engineering","location":"Lisbon Portugal","acronym":"NLBSE '24","sponsor":["SIGSOFT ACM Special Interest Group on Software Engineering","IEEE CS","Faculty of Engineering of University of Porto"]},"container-title":["Proceedings of the Third ACM\/IEEE International Workshop on NL-based Software Engineering"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3643787.3648038","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3643787.3648038","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T20:57:45Z","timestamp":1750280265000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3643787.3648038"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,4,20]]},"references-count":53,"alternative-id":["10.1145\/3643787.3648038","10.1145\/3643787"],"URL":"https:\/\/doi.org\/10.1145\/3643787.3648038","relation":{},"subject":[],"published":{"date-parts":[[2024,4,20]]},"assertion":[{"value":"2024-08-26","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}