{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,23]],"date-time":"2026-06-23T08:48:26Z","timestamp":1782204506565,"version":"3.54.5"},"reference-count":59,"publisher":"Springer Science and Business Media LLC","issue":"5","license":[{"start":{"date-parts":[[2026,5,9]],"date-time":"2026-05-09T00:00:00Z","timestamp":1778284800000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2026,5,9]],"date-time":"2026-05-09T00:00:00Z","timestamp":1778284800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100020838","name":"Pontif\u00edcia Universidade Cat\u00f3lica Do Rio De Janeiro","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100020838","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Empir Software Eng"],"published-print":{"date-parts":[[2026,9]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>Social coding platforms like GitHub facilitate collaborative software development through pull requests (PRs), which generate discussions that significantly impact code quality, requirements, and design. Such conversations become a rich source of insights for improving development practices and predicting project outcomes and are subject to several human aspects that have been linked to code quality and PR acceptance. Sentiment analysis is one of the many ways to try to understand these human aspects. However, PR discussions are multifaceted, often involving technical jargon and aspects which limits the utility of general-purpose sentiment analysis tools. This has led to the creation of SE-specific tools, but recent studies have also observed that they demonstrate limited effectiveness. Thus, this study explores the potential of using large language models (LLMs) for this purpose, given their enhanced contextual understanding and ability to process technical language. We evaluated ten LLMs across proprietary and open-source categories, using two complementary datasets: a curated Gold dataset and the PRemo dataset, which captures real-world PR discussions. The models were assessed under zero-shot, few-shot and chain-of-thought prompting techniques on 8,913 messages. In addition, we establish baselines by evaluating fine-tuned transformer-based models. Results show that GPT-4o achieved the highest overall performance across the LLMs, though smaller models, such as Mistral Small and Deepseek-R1 32B delivered competitive results. Transformer-based models achieved excellent performance on the Gold dataset but exhibited degradation on the PRemo dataset. Finally, we conducted a qualitative analysis of misclassified instances, revealing recurring challenges related to technical terminology, sentiment-charged keywords, message length, and contextual ambiguity. These findings suggest that model selection should balance performance requirements against practical constraints, rather than defaulting to the largest available models.<\/jats:p>","DOI":"10.1007\/s10664-026-10868-6","type":"journal-article","created":{"date-parts":[[2026,5,9]],"date-time":"2026-05-09T05:23:49Z","timestamp":1778304229000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Leveraging large language models for sentiment analysis in GitHub pull request discussions"],"prefix":"10.1007","volume":"31","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-4226-2458","authenticated-orcid":false,"given":"Daniel","family":"Coutinho","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0007-9447-9858","authenticated-orcid":false,"given":"Breno","family":"Braga","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0008-3496-4197","authenticated-orcid":false,"given":"Theo","family":"Canuto","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0799-2829","authenticated-orcid":false,"given":"Juliana Alves","family":"Pereira","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7557-9091","authenticated-orcid":false,"given":"Wesley K. G.","family":"Assun\u00e7\u00e3o","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0612-5790","authenticated-orcid":false,"given":"Igor","family":"Steinmacher","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1399-7535","authenticated-orcid":false,"given":"Marco","family":"Gerosa","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5788-5215","authenticated-orcid":false,"given":"Alessandro","family":"Garcia","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2026,5,9]]},"reference":[{"key":"10868_CR1","doi-asserted-by":"publisher","unstructured":"Ain QT, Ali M, Riaz A, Noureen A, Kamran M, Hayat B, Rehman A (2017) Sentiment analysis using deep learning techniques: a review. Intern J Adv Comput Sci Appl 8(6) https:\/\/doi.org\/10.14569\/IJACSA.2017.080657","DOI":"10.14569\/IJACSA.2017.080657"},{"key":"10868_CR2","doi-asserted-by":"publisher","unstructured":"Barbosa C, Uch\u00f4a A, Coutinho D, Assun\u00e7ao WK, Oliveira A, Garcia A, Fonseca B, Rabelo M, Coelho JE, Carvalho E et\u00a0al (2023) Beyond the code: Investigating the effects of pull request conversations on design decay. In: 2023 ACM\/IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM), IEEE, pp 1\u201312 https:\/\/doi.org\/10.1109\/ESEM56168.2023.10304805","DOI":"10.1109\/ESEM56168.2023.10304805"},{"key":"10868_CR3","unstructured":"Brassard-Gourdeau \u00c9, Khoury R (2018) Impact of sentiment detection to recognize toxic and subversive online comments. arXiv preprint arXiv:1812.01704"},{"key":"10868_CR4","doi-asserted-by":"publisher","unstructured":"Brassard-Gourdeau E, Khoury R (2019) Subversive toxicity detection using sentiment information. In: Proceedings of the third workshop on abusive language online, pp 1\u201310 https:\/\/doi.org\/10.18653\/v1\/W19-3501","DOI":"10.18653\/v1\/W19-3501"},{"key":"10868_CR5","first-page":"1877","volume":"33","author":"T Brown","year":"2020","unstructured":"Brown T, Mann B, Ryder N, Subbiah M, Kaplan JD, Dhariwal P, Neelakantan A, Shyam P, Sastry G, Askell A et al (2020) Language models are few-shot learners. Adv Neural Inf Process Syst 33:1877\u20131901","journal-title":"Adv Neural Inf Process Syst"},{"key":"10868_CR6","doi-asserted-by":"publisher","unstructured":"Calefato F, Lanubile F, Novielli N (2017) Emotxt: A toolkit for emotion recognition from text. In: Proceedings of the seventh international conference on affective computing and intelligent interaction, IEEE, pp 79\u201385 https:\/\/doi.org\/10.1109\/ACIIW.2017.8272591","DOI":"10.1109\/ACIIW.2017.8272591"},{"key":"10868_CR7","doi-asserted-by":"publisher","unstructured":"Calefato F, Lanubile F, Maiorano F, Novielli N (2018) Sentiment polarity detection for software development. In: Proceedings of the 40th international conference on software engineering, IEEE, pp 128-128 https:\/\/doi.org\/10.1145\/3180155.3182519","DOI":"10.1145\/3180155.3182519"},{"key":"10868_CR8","doi-asserted-by":"publisher","unstructured":"Coutinho D, Cito L, Lima MV, Arantes B, Pereira JA, Arriel J, Godinho J, Martins V, Lib\u00f3rio P, Leite L, Garcia A, Assun\u00e7\u00e3o WKG, Steinmacher I, Baffa A, Fonseca B (2024) \"looks good to me ;-)\": Assessing sentiment analysis tools for pull request discussions. In: Proceedings of the 28th International Conference on Evaluation and Assessment in Software Engineering (EASE 2024), ACM, Salerno, Italy, p\u00a011. https:\/\/doi.org\/10.1145\/3661167.3661189","DOI":"10.1145\/3661167.3661189"},{"key":"10868_CR9","unstructured":"Coutinho D, Braga B, Canuto T, Pereira JA, Assun\u00e7\u00e3o WKG, Steinmacher I, Gerosa M, Garcia A (2025a) Replication package. https:\/\/github.com\/opus-research\/LLM4SA-replication"},{"key":"10868_CR10","doi-asserted-by":"publisher","unstructured":"Coutinho D, Pereira JA, Neves B, Correia JLM, da\u00a0Silva CBV, Assun\u00e7\u00e3o WKG, Steinmacher I, Gerosa MA, Baffa A, Garcia A (2025b) PRemo: A Dataset of Emotions Found on Pull Request Discussions. In: Proceedings of the 39th Brazilian Symposium on Software Engineering (SBES), SBC, Brazil, to appear https:\/\/doi.org\/10.5753\/sbes.2025.9936","DOI":"10.5753\/sbes.2025.9936"},{"issue":"3","key":"10868_CR11","doi-asserted-by":"publisher","first-page":"483","DOI":"10.3390\/electronics9030483","volume":"9","author":"NC Dang","year":"2020","unstructured":"Dang NC, Moreno-Garc\u00eda MN, De la Prieta F (2020) Sentiment analysis based on deep learning: A comparative study. Electronics 9(3):483 https:\/\/doi.org\/10.3390\/electronics9030483","journal-title":"Electronics"},{"key":"10868_CR12","unstructured":"Devlin J, Chang MW, Lee K, Toutanova K (2018) Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805"},{"key":"10868_CR13","unstructured":"Diao S, Wang P, Lin Y, Pan R, Liu X, Zhang T (2023) Active prompting with chain-of-thought for large language models. arXiv preprint arXiv:2302.12246"},{"issue":"6","key":"10868_CR14","doi-asserted-by":"publisher","first-page":"3790","DOI":"10.1007\/s10664-019-09700-1","volume":"24","author":"M El Mezouar","year":"2019","unstructured":"El Mezouar M, Zhang F, Zou Y (2019) An empirical study on the teams structures in social coding using github projects. Empir Softw Eng 24(6):3790\u20133823 https:\/\/doi.org\/10.1007\/s10664-019-09700-1","journal-title":"Empir Softw Eng"},{"key":"10868_CR15","doi-asserted-by":"publisher","first-page":"e289","DOI":"10.7717\/peerj.289","volume":"2","author":"D Graziotin","year":"2014","unstructured":"Graziotin D, Wang X, Abrahamsson P (2014) Happy software developers solve problems better: psychological measurements in empirical software engineering. PeerJ 2:e289 https:\/\/doi.org\/10.7717\/peerj.289","journal-title":"PeerJ"},{"key":"10868_CR16","doi-asserted-by":"publisher","first-page":"e18","DOI":"10.7717\/peerj-cs.18","volume":"1","author":"D Graziotin","year":"2015","unstructured":"Graziotin D, Wang X, Abrahamsson P (2015) How do you feel, developer? an explanatory theory of the impact of affects on programming performance. PeerJ Comput Sci 1:e18 https:\/\/doi.org\/10.7717\/peerj-cs.18","journal-title":"PeerJ Comput Sci"},{"key":"10868_CR17","doi-asserted-by":"crossref","unstructured":"Gururangan S, Marasovic A, Swayamdipta S, Lo K, Beltagy I, Downey D, Smith NA (2020) Don\u2019t stop pretraining: Adapt language models to domains and tasks. arXiv preprint arXiv:2004.10964","DOI":"10.18653\/v1\/2020.acl-main.740"},{"key":"10868_CR18","doi-asserted-by":"publisher","unstructured":"Guzman E, Bruegge B (2013) Towards emotional awareness in software development teams. In: Proceedings of the 2013 9th joint meeting on foundations of software engineering, pp 671\u2013674 https:\/\/doi.org\/10.1145\/2491411.2494578","DOI":"10.1145\/2491411.2494578"},{"key":"10868_CR19","doi-asserted-by":"publisher","unstructured":"Guzman E, Az\u00f3car D, Li Y (2014) Sentiment analysis of commit comments in github: An empirical study. In: Proceedings of the 11th working conference on mining software repositories, ACM, pp 352\u2013355 https:\/\/doi.org\/10.1145\/2597073.2597118","DOI":"10.1145\/2597073.2597118"},{"key":"10868_CR20","doi-asserted-by":"crossref","unstructured":"Hasan MA, Das S, Anjum A, Alam F, Anjum A, Sarker A, Noori SRH (2024) Zero- and few-shot prompting with llms: A comparative study with fine-tuned models for bangla sentiment analysis. arXiv preprint arXiv:2308.10783v2","DOI":"10.63317\/3edi4825svh3"},{"issue":"1","key":"10868_CR21","doi-asserted-by":"publisher","first-page":"3","DOI":"10.1007\/s10664-021-10058-6","volume":"27","author":"H Hata","year":"2022","unstructured":"Hata H, Novielli N, Baltes S, Kula RG, Treude C (2022) Github discussions: An exploratory study of early adoption. Empir Softw Eng 27(1):3 https:\/\/doi.org\/10.1007\/s10664-021-10058-6","journal-title":"Empir Softw Eng"},{"key":"10868_CR22","doi-asserted-by":"publisher","unstructured":"Herrmann M, Kl\u00fcnder J (2021) From textual to verbal communication: Towards applying sentiment analysis to a software project meeting. In: 2021 Fourth International Workshop on Affective Computing for Requirements Engineering (AffectRE). IEEE https:\/\/doi.org\/10.1109\/REW53955.2021.00065","DOI":"10.1109\/REW53955.2021.00065"},{"key":"10868_CR23","doi-asserted-by":"publisher","unstructured":"Imran MM, Chatterjee P, Damevski K (2024) Uncovering the causes of emotions in software developer communication using zero-shot llms. In: Proceedings of the IEEE\/ACM 46th international conference on software engineering, pp 1\u201313 https:\/\/doi.org\/10.1145\/3597503.3639223","DOI":"10.1145\/3597503.3639223"},{"key":"10868_CR24","doi-asserted-by":"publisher","unstructured":"Islam MR, Zibran MF (2017) Leveraging automated sentiment analysis in software engineering. In: 2017 IEEE\/ACM 14th International Conference on Mining Software Repositories (MSR), pp 203\u2013214. https:\/\/doi.org\/10.1109\/MSR.2017.9","DOI":"10.1109\/MSR.2017.9"},{"key":"10868_CR26","doi-asserted-by":"publisher","first-page":"125","DOI":"10.1016\/j.jss.2018.08.030","volume":"145","author":"MR Islam","year":"2018","unstructured":"Islam MR, Zibran MF (2018) Sentistrength-se: Exploiting domain specificity for improved sentiment analysis in software engineering text. J Syst Softw 145:125\u2013146 https:\/\/doi.org\/10.1016\/j.jss.2018.08.030","journal-title":"J Syst Softw"},{"key":"10868_CR27","doi-asserted-by":"publisher","unstructured":"Kalliamvakou E, Damian D, Blincoe K, Singer L, German DM (2015) Open source-style collaborative development practices in commercial projects using github. In: 2015 IEEE\/ACM 37th IEEE international conference on software engineering, IEEE, vol\u00a01, pp 574\u2013585 https:\/\/doi.org\/10.1109\/ICSE.2015.74","DOI":"10.1109\/ICSE.2015.74"},{"key":"10868_CR28","unstructured":"Kolchyna O, Souza TT, Treleaven P, Aste T (2015) Twitter sentiment analysis: Lexicon method, machine learning method and their combination. arXiv preprint arXiv:1507.00955"},{"key":"10868_CR29","doi-asserted-by":"crossref","unstructured":"Li Z, Peng B, He P, Yan X (2023) Evaluating the instruction-following robustness of large language models to prompt injection. arXiv preprint arXiv:2308.10819","DOI":"10.18653\/v1\/2024.emnlp-main.33"},{"key":"10868_CR30","doi-asserted-by":"publisher","unstructured":"Lin B, Zampetti F, Bavota G, Di\u00a0Penta M, Lanza M, Oliveto R (2018) Sentiment analysis for software engineering: How far can we go? In: Proceedings of the 40th International Conference on Software Engineering, Association for Computing Machinery, New York, NY, USA, ICSE \u201918, pp 94\u2013104. https:\/\/doi.org\/10.1145\/3180155.3180195","DOI":"10.1145\/3180155.3180195"},{"key":"10868_CR31","doi-asserted-by":"publisher","unstructured":"Liu B (2012) Sentiment analysis and opinion mining. Springer Nature https:\/\/doi.org\/10.1162\/COLI_r_00186","DOI":"10.1162\/COLI_r_00186"},{"key":"10868_CR32","doi-asserted-by":"publisher","unstructured":"Mannan UA, Ahmed I, Jensen C, Sarma A (2020) On the relationship between design discussions and design quality: A case study of apache projects. In: Proceedings of the 28th ACM joint meeting on european software engineering conference and symposium on the foundations of software engineering, pp 543\u2013553 https:\/\/doi.org\/10.1145\/3368089.3409707","DOI":"10.1145\/3368089.3409707"},{"key":"10868_CR33","doi-asserted-by":"publisher","unstructured":"Marvin G, Hellen N, Jjingo D, Nakatumba-Nabende J (2023) Prompt engineering in large language models. In: International conference on data intelligence and cognitive informatics, Springer, pp 387\u2013402 https:\/\/doi.org\/10.1007\/978-981-99-7962-2_30","DOI":"10.1007\/978-981-99-7962-2_30"},{"key":"10868_CR34","unstructured":"Niimi J (2024) Dynamic sentiment analysis with local large language models using majority voting: A study on factors affecting restaurant evaluation. arXiv preprint arXiv:2407.13069"},{"key":"10868_CR35","doi-asserted-by":"publisher","unstructured":"Novielli N, Calefato F, Dongiovanni D, Girardi D, Lanubile F (2020) Can we use se-specific sentiment analysis tools in a cross-platform setting? In: Proceedings of the 17th international conference on mining software repositories, association for computing machinery, New York, NY, USA, MSR \u201920, pp 158\u2013168. https:\/\/doi.org\/10.1145\/3379597.3387446","DOI":"10.1145\/3379597.3387446"},{"key":"10868_CR36","doi-asserted-by":"publisher","unstructured":"Novielli N, Girardi D, Lanubile F (2018) A benchmark study on sentiment analysis for software engineering research. In: Proceedings of the 15th international conference on mining software repositories, pp 364\u2013375 https:\/\/doi.org\/10.1145\/3196398.3196403","DOI":"10.1145\/3196398.3196403"},{"key":"10868_CR37","doi-asserted-by":"publisher","unstructured":"Obaidi M, Herrmann M, Schneider K, Kl\u00fcnder J (2025a) Towards trustworthy sentiment analysis in software engineering: Dataset characteristics and tool selection. In: 2025 IEEE 33rd International Requirements Engineering Conference Workshops (REW), IEEE, pp 538\u2013547 https:\/\/doi.org\/10.1109\/REW66121.2025.00080","DOI":"10.1109\/REW66121.2025.00080"},{"key":"10868_CR38","doi-asserted-by":"publisher","unstructured":"Obaidi M, Herrmann M, Schmid E, Ochsner R, Schneider K, Kl\u00fcnder J (2025b) A german gold-standard dataset for sentiment analysis in software engineering. In: 2025 IEEE 33rd International Requirements Engineering Conference Workshops (REW), IEEE, pp 107\u2013114 https:\/\/doi.org\/10.1109\/REW66121.2025.00018","DOI":"10.1109\/REW66121.2025.00018"},{"key":"10868_CR39","doi-asserted-by":"publisher","unstructured":"Obaidi M, Nagel L, Specht A, Kl\u00fcnder J (2022) Sentiment analysis tools in software engineering: A systematic mapping study. Inform Softw Technol 107018 https:\/\/doi.org\/10.1016\/j.infsof.2022.107018","DOI":"10.1016\/j.infsof.2022.107018"},{"key":"10868_CR40","doi-asserted-by":"publisher","unstructured":"Ortu M, Murgia A, Destefanis G, Tourani P, Tonelli R, Marchesi M, Adams B (2016) The emotional side of software developers in jira. In: Proceedings of the 13th International Conference on Mining Software Repositories, Association for Computing Machinery, New York, NY, USA, MSR \u201916, pp 480\u2013483. https:\/\/doi.org\/10.1145\/2901739.2903505, https:\/\/doi.org\/10.1145\/2901739.2903505","DOI":"10.1145\/2901739.2903505"},{"key":"10868_CR41","doi-asserted-by":"crossref","unstructured":"Pang B, Lee L et\u00a0al (2008) Opinion mining and sentiment analysis. Found Trends\u00ae Inf Retriev 2(1\u20132):1\u2013135","DOI":"10.1561\/1500000011"},{"key":"10868_CR42","doi-asserted-by":"publisher","unstructured":"Rahman MM, Roy CK (2014) An insight into the pull requests of github. In: 11th working conference on mining software repositories, pp 364\u2013367 https:\/\/doi.org\/10.1145\/2597073.2597121","DOI":"10.1145\/2597073.2597121"},{"key":"10868_CR43","doi-asserted-by":"publisher","unstructured":"Sayago-Heredia J, Chango G, P\u00e9rez-Castillo R, Piattini M (2022) Exploring the impact of toxic comments in code quality. In: ENASE, pp 335\u2013343 https:\/\/doi.org\/10.5220\/0011039700003176","DOI":"10.5220\/0011039700003176"},{"key":"10868_CR44","doi-asserted-by":"publisher","unstructured":"Shafikuzzaman M, Islam MR, Rolli AC, Akhter S, Seliya N (2024) An empirical evaluation of the zero-shot, few-shot, and traditional fine-tuning based pretrained language models for sentiment analysis in software engineering. IEEE Access https:\/\/doi.org\/10.1109\/ACCESS.2024.3439450","DOI":"10.1109\/ACCESS.2024.3439450"},{"issue":"6","key":"10868_CR45","doi-asserted-by":"publisher","first-page":"1061","DOI":"10.1037\/0022-3514.52.6.1061","volume":"52","author":"P Shaver","year":"1987","unstructured":"Shaver P, Schwartz J, Kirson D, O\u2019connor C (1987) Emotion knowledge: further exploration of a prototype approach. J Pers Soc Psychol 52(6):1061 https:\/\/doi.org\/10.1037\/0022-3514.52.6.1061","journal-title":"J Pers Soc Psychol"},{"key":"10868_CR46","doi-asserted-by":"publisher","first-page":"110416","DOI":"10.1016\/j.jss.2019.110416","volume":"158","author":"J Tantisuwankul","year":"2019","unstructured":"Tantisuwankul J, Nugroho YS, Kula RG, Hata H, Rungsawang A, Leelaprute P, Matsumoto K (2019) A topological analysis of communication channels for knowledge sharing in contemporary github projects. J Syst Softw 158:110416 https:\/\/doi.org\/10.1016\/j.jss.2019.110416","journal-title":"J Syst Softw"},{"issue":"1","key":"10868_CR47","doi-asserted-by":"publisher","first-page":"163","DOI":"10.1002\/asi.21662","volume":"63","author":"M Thelwall","year":"2012","unstructured":"Thelwall M, Buckley K, Paltoglou G (2012) Sentiment strength detection for the social web. J Am Soc Inform Sci Technol 63(1):163\u201317. https:\/\/doi.org\/10.1002\/asi.21662","journal-title":"J Am Soc Inform Sci Technol"},{"key":"10868_CR48","doi-asserted-by":"publisher","unstructured":"Tsay J, Dabbish L, Herbsleb J (2014a) Influence of social and technical factors for evaluating contribution in github. In: Proceedings of the 36th international conference on Software engineering, pp 356\u2013366 https:\/\/doi.org\/10.1145\/2568225.2568315","DOI":"10.1145\/2568225.2568315"},{"key":"10868_CR49","doi-asserted-by":"publisher","unstructured":"Tsay J, Dabbish L, Herbsleb J (2014b) Let\u2019s talk about it: Evaluating contributions through discussion in github. In: Proceedings of the 22nd ACM SIGSOFT international symposium on foundations of software engineering, association for computing machinery, New York, NY, USA, FSE 2014, pp 144\u2013154. https:\/\/doi.org\/10.1145\/2635868.2635882","DOI":"10.1145\/2635868.2635882"},{"key":"10868_CR50","unstructured":"Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser \u0141, Polosukhin I (2017) Attention is all you need. Adv Neural Inf Process Syst 30"},{"issue":"7","key":"10868_CR51","doi-asserted-by":"publisher","first-page":"1402","DOI":"10.1109\/TSE.2019.2924006","volume":"47","author":"G Viviani","year":"2019","unstructured":"Viviani G, Famelis M, Xia X, Janik-Jones C, Murphy GC (2019) Locating latent design information in developer discussions: A study on pull requests. IEEE Trans Software Eng 47(7):1402\u20131413 https:\/\/doi.org\/10.1109\/TSE.2019.2924006","journal-title":"IEEE Trans Software Eng"},{"key":"10868_CR52","doi-asserted-by":"crossref","unstructured":"Wei J, Wang X, Schuurmans D, Bosma M, Ichter B, Xia F, Chi EH, Le QV, Zhou D (2022) Chain of thought prompting elicits reasoning in large language models. arXiv preprint arXiv:2201.11903","DOI":"10.52202\/068431-1800"},{"key":"10868_CR53","doi-asserted-by":"publisher","unstructured":"Wohlin C, Runeson P, H\u00f6st M, Ohlsson M, Regnell B, Wessl\u00e9n A (2012) Experimentation in software engineering, 1st edn. Springer Science & Business Media https:\/\/doi.org\/10.1007\/978-3-642-29044-2","DOI":"10.1007\/978-3-642-29044-2"},{"key":"10868_CR54","unstructured":"Xing F (2024) Designing heterogeneous llm agents for financial sentiment analysis. arXiv preprint arXiv:2401.05799"},{"key":"10868_CR55","doi-asserted-by":"crossref","unstructured":"Zhan T, Shi C, Shi Y, Li H, Lin Y (2024) Optimization techniques for sentiment analysis based on llm (gpt-3). arXiv preprint arXiv:2405.09770","DOI":"10.54254\/2755-2721\/77\/2024MA0060"},{"key":"10868_CR56","doi-asserted-by":"publisher","unstructured":"Zhang T, Xu B, Thung F, Haryono SA, Lo D, Jiang L (2020) Sentiment analysis for software engineering: How far can pre-trained transformer models go? In: 2020 IEEE International Conference on Software Maintenance and Evolution (ICSME), IEEE, pp 70\u201380 https:\/\/doi.org\/10.1109\/ICSME46990.2020.00017","DOI":"10.1109\/ICSME46990.2020.00017"},{"issue":"2","key":"10868_CR57","doi-asserted-by":"publisher","first-page":"849","DOI":"10.1109\/TSE.2022.3165056","volume":"49","author":"X Zhang","year":"2022","unstructured":"Zhang X, Yu Y, Gousios G, Rastogi A (2022) Pull request decisions explained: An empirical overview. IEEE Trans Software Eng 49(2):849\u2013871 https:\/\/doi.org\/10.1109\/TSE.2022.3165056","journal-title":"IEEE Trans Software Eng"},{"key":"10868_CR59","doi-asserted-by":"publisher","unstructured":"Zhang T, Irsan IC, Thung F, Lo D (2025) Revisiting sentiment analysis for software engineering in the era of large language models. IEEE Trans Software Eng 34(3): 1-30 https:\/\/doi.org\/10.1145\/3697009","DOI":"10.1145\/3697009"},{"key":"10868_CR60","unstructured":"Zhou Y, Muresanu AI, Han Z, Paster K, Pitis S, Chan H, Ba J (2022) Large language models are human-level prompt engineers. arXiv preprint arXiv:2211.01910"},{"key":"10868_CR61","doi-asserted-by":"publisher","unstructured":"Zhu K, Wang J, Zhou J, Wang Z, Chen H, Wang Y, Yang L, Ye W, Zhang Y, Gong N et\u00a0al (2023) Promptrobust: Towards evaluating the robustness of large language models on adversarial prompts. In: Proceedings of the 1st ACM workshop on large AI systems and models with privacy and safety analysis, pp 57\u201368 https:\/\/doi.org\/10.1145\/3689217.3690621","DOI":"10.1145\/3689217.3690621"}],"container-title":["Empirical Software Engineering"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10664-026-10868-6.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10664-026-10868-6","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10664-026-10868-6.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,6,23]],"date-time":"2026-06-23T08:01:46Z","timestamp":1782201706000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10664-026-10868-6"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,5,9]]},"references-count":59,"journal-issue":{"issue":"5","published-print":{"date-parts":[[2026,9]]}},"alternative-id":["10868"],"URL":"https:\/\/doi.org\/10.1007\/s10664-026-10868-6","relation":{},"ISSN":["1382-3256","1573-7616"],"issn-type":[{"value":"1382-3256","type":"print"},{"value":"1573-7616","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,5,9]]},"assertion":[{"value":"14 August 2025","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"21 April 2026","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"9 May 2026","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"This item is not applicable.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethical Approval"}},{"value":"This item is not applicable.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Informed Consent"}},{"value":"The authors declare that they have no conflict of interest.","order":4,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of Interest"}},{"value":"This item is not applicable.","order":5,"name":"Ethics","group":{"name":"EthicsHeading","label":"Clinical Trial Number"}}],"article-number":"140"}}