{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,11,4]],"date-time":"2025-11-04T10:56:58Z","timestamp":1762253818460,"version":"3.40.5"},"reference-count":45,"publisher":"Springer Science and Business Media LLC","issue":"4","license":[{"start":{"date-parts":[[2021,6,5]],"date-time":"2021-06-05T00:00:00Z","timestamp":1622851200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2021,6,5]],"date-time":"2021-06-05T00:00:00Z","timestamp":1622851200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"name":"Universit\u00e0 degli Studi di Bari Aldo Moro"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Empir Software Eng"],"published-print":{"date-parts":[[2021,7]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Sentiment analysis methods have become popular for investigating human communication, including discussions related to software projects. Since general-purpose sentiment analysis tools do not fit well with the information exchanged by software developers, new tools, specific for software engineering (SE), have been developed. We investigate to what extent off-the-shelf SE-specific tools for sentiment analysis mitigate the threats to conclusion validity of empirical studies in software engineering, highlighted by previous research. First, we replicate two studies addressing the role of sentiment in security discussions on GitHub and in question-writing on Stack Overflow. Then, we extend the previous studies by assessing to what extent the tools agree with each other and with the manual annotation on a gold standard of 600 documents. We find that different SE-specific sentiment analysis tools might lead to contradictory results at a fine-grain level, when used <jats:italic>off-the-shelf<\/jats:italic>. Conversely, platform-specific tuning or retraining might be needed to take into account differences in platform conventions, jargon, or document lengths.<\/jats:p>","DOI":"10.1007\/s10664-021-09960-w","type":"journal-article","created":{"date-parts":[[2021,6,5]],"date-time":"2021-06-05T11:02:38Z","timestamp":1622890958000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":21,"title":["Assessment of off-the-shelf SE-specific sentiment analysis tools: An extended replication study"],"prefix":"10.1007","volume":"26","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-1160-2608","authenticated-orcid":false,"given":"Nicole","family":"Novielli","sequence":"first","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2654-1588","authenticated-orcid":false,"given":"Fabio","family":"Calefato","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3373-7589","authenticated-orcid":false,"given":"Filippo","family":"Lanubile","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1418-0095","authenticated-orcid":false,"given":"Alexander","family":"Serebrenik","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2021,6,5]]},"reference":[{"key":"9960_CR1","doi-asserted-by":"crossref","unstructured":"Ahmed T, Bosu A, Iqbal A, Rahimi S (2017) Senticr: A customized sentiment analysis tool for code review interactions. In: Proceedings of the 32nd IEEE\/ACM international conference on automated software engineering. ASE 2017. IEEE Press, Piscataway, pp 106\u2013111, http:\/\/dl.acm.org\/citation.cfm?id=3155562.3155579","DOI":"10.1109\/ASE.2017.8115623"},{"key":"9960_CR2","doi-asserted-by":"publisher","unstructured":"Blaz CCA, Becker K (2016) Sentiment analysis in tickets for it support. In: Proceedings of the 13th international conference on mining software repositories. MSR \u201916. ACM, New York, pp 235\u2013246, https:\/\/doi.org\/10.1145\/2901739.2901781","DOI":"10.1145\/2901739.2901781"},{"issue":"2","key":"9960_CR3","doi-asserted-by":"publisher","first-page":"55","DOI":"10.1109\/MIS.2013.28","volume":"28","author":"C Bosco","year":"2013","unstructured":"Bosco C, Patti V, Bolioli A (2013) Developing corpora for sentiment analysis: the case of irony and senti-tut. IEEE Intell Syst 28(2):55\u201363","journal-title":"IEEE Intell Syst"},{"issue":"3","key":"9960_CR4","doi-asserted-by":"publisher","first-page":"1352","DOI":"10.1007\/s10664-017-9546-9","volume":"23","author":"F Calefato","year":"2018","unstructured":"Calefato F, Lanubile F, Maiorano F, Novielli N (2018a) Sentiment polarity detection for software development. Empirical Softw Engg 23(3):1352\u20131382. https:\/\/doi.org\/10.1007\/s10664-017-9546-9","journal-title":"Empirical Softw Engg"},{"key":"9960_CR5","doi-asserted-by":"publisher","first-page":"186","DOI":"10.1016\/j.infsof.2017.10.009","volume":"94","author":"F Calefato","year":"2018","unstructured":"Calefato F, Lanubile F, Novielli N (2018b) How to ask for technical help? evidence-based guidelines for writing questions on stack overflow. Inform Softw Technol 94:186\u2013207. https:\/\/doi.org\/10.1016\/j.infsof.2017.10.009, cited By 10","journal-title":"Inform Softw Technol"},{"key":"9960_CR6","unstructured":"Carver JC (2010) Towards reporting guidelines for experimental replications: A proposal. In: 1st International workshop on replication in empirical software engineering research"},{"key":"9960_CR7","doi-asserted-by":"publisher","first-page":"321","DOI":"10.1613\/jair.953","volume":"16","author":"NV Chawla","year":"2002","unstructured":"Chawla NV, Bowyer KW, Hall LO, Kegelmeyer WP (2002) Smote: Synthetic minority over-sampling technique. J Artif Intell Res 16:321\u2013357","journal-title":"J Artif Intell Res"},{"key":"9960_CR8","doi-asserted-by":"publisher","unstructured":"Chen Z, Cao Y, Lu X, Mei Q, Liu X (2019) Sentimoji: An emoji-powered learning approach for sentiment analysis in software engineering. In: Proceedings of the 2019 27th ACM joint meeting on european software engineering conference and symposium on the foundations of software engineering, association for computing machinery, New York, NY, USA, ESEC\/FSE 2019, pp 841\u2013852. https:\/\/doi.org\/10.1145\/3338906.3338977","DOI":"10.1145\/3338906.3338977"},{"key":"9960_CR9","doi-asserted-by":"publisher","first-page":"213","DOI":"10.1037\/h0026256","volume":"70","author":"J Cohen","year":"1968","unstructured":"Cohen J (1968) Weighted kappa: Nominal scale agreement with provision for scaled disagreement or partial credit. Psychol Bull 70:213\u2013220","journal-title":"Psychol Bull"},{"issue":"1","key":"9960_CR10","doi-asserted-by":"publisher","first-page":"95","DOI":"10.1162\/089120104773633402","volume":"30","author":"M Glass","year":"2004","unstructured":"Di Eugenio B, Glass M (2004) The kappa statistic: A second look. Computat Linguist 30 (1):95\u2013101. https:\/\/doi.org\/10.1162\/089120104773633402","journal-title":"Computat Linguist"},{"key":"9960_CR11","doi-asserted-by":"publisher","unstructured":"Ebert F, Castor F, Novielli N, Serebrenik A (2018) Communicative intention in code review questions. In: 2018 IEEE international conference on software maintenance and evolution, ICSME 2018, madrid, spain, september 23-29, 2018, pp 519\u2013523. https:\/\/doi.org\/10.1109\/ICSME.2018.00061","DOI":"10.1109\/ICSME.2018.00061"},{"key":"9960_CR12","doi-asserted-by":"publisher","unstructured":"Gachechiladze D, Lanubile F, Novielli N, Serebrenik A (2017) Anger and its direction in collaborative software development. In: Proceedings of the 39th international conference on software engineering: New ideas and emerging results track. IEEE Press, Piscataway, NJ, USA, ICSE-NIER \u201917, pp 11\u201314. https:\/\/doi.org\/10.1109\/ICSE-NIER.2017.18","DOI":"10.1109\/ICSE-NIER.2017.18"},{"key":"9960_CR13","doi-asserted-by":"crossref","unstructured":"Gousios G (2013) The ghtorent dataset and tool suite. In: Proceedings of the 10th working conference on mining software repositories, IEEE Press, Piscataway, NJ, USA, MSR \u201913, pp 233\u2013236. http:\/\/dl.acm.org\/citation.cfm?id=2487085.2487132","DOI":"10.1109\/MSR.2013.6624034"},{"key":"9960_CR14","doi-asserted-by":"publisher","unstructured":"Guzman E, Az\u00f3car D, Li Y (2014) Sentiment analysis of commit comments in github: An empirical study. In: Proceedings of the 11th working conference on mining software repositories, ACM, New York, NY, USA, MSR 2014, pp 352\u2013355. https:\/\/doi.org\/10.1145\/2597073.2597118","DOI":"10.1145\/2597073.2597118"},{"key":"9960_CR15","doi-asserted-by":"publisher","unstructured":"Guzman E, Alkadhi R, Seyff N (2016) A needle in a haystack: What do twitter users say about software?. In: 24Th IEEE international requirements engineering conference, RE 2016, beijing, china, september 12-16, 2016, pp 96\u2013105. https:\/\/doi.org\/10.1109\/RE.2016.67","DOI":"10.1109\/RE.2016.67"},{"key":"9960_CR16","doi-asserted-by":"publisher","unstructured":"Imtiaz N, Middleton J, Girouard P, Murphy-Hill E (2018) Sentiment and politeness analysis tools on developer discussions are unreliable, but so are people. In: Proceedings of the 3rd international workshop on emotion awareness in software engineering, ACM, New York, NY, USA, SEmotion \u201918, pp 55\u201361. https:\/\/doi.org\/10.1145\/3194932.3194938","DOI":"10.1145\/3194932.3194938"},{"key":"9960_CR17","doi-asserted-by":"publisher","unstructured":"Islam MR, Zibran MF (2017) Leveraging automated sentiment analysis in software engineering. In: Proceedings of the 14th international conference on mining software repositories, IEEE Press, Piscataway, NJ, USA, MSR \u201917, pp 203\u2013214. https:\/\/doi.org\/10.1109\/MSR.2017.9","DOI":"10.1109\/MSR.2017.9"},{"key":"9960_CR18","doi-asserted-by":"crossref","unstructured":"Islam MR, Zibran MF (2018a) A comparison of software engineering domain specific sentiment analysis tools. In: 2018 IEEE 25th international conference on software analysis, evolution and reengineering (SANER), pp 487\u2013491","DOI":"10.1109\/SANER.2018.8330245"},{"key":"9960_CR19","doi-asserted-by":"publisher","unstructured":"Islam MR, Zibran MF (2018b) Deva: Sensing emotions in the valence arousal space in software engineering text. In: Proceedings of the 33rd Annual ACM Symposium on Applied Computing, ACM, New York, NY, USA, SAC \u201918, pp 1536\u20131543. https:\/\/doi.org\/10.1145\/3167132.3167296","DOI":"10.1145\/3167132.3167296"},{"issue":"5","key":"9960_CR20","doi-asserted-by":"publisher","first-page":"2543","DOI":"10.1007\/s10664-016-9493-x","volume":"22","author":"R Jongeling","year":"2017","unstructured":"Jongeling R, Sarkar P, Datta S, Serebrenik A (2017) On negative results when using sentiment analysis tools for software engineering research. Empirical Softw Engg 22(5):2543\u20132584. https:\/\/doi.org\/10.1007\/s10664-016-9493-x","journal-title":"Empirical Softw Engg"},{"issue":"3","key":"9960_CR21","doi-asserted-by":"publisher","first-page":"357","DOI":"10.1007\/s00766-018-0293-2","volume":"23","author":"Z Kurtanovic","year":"2018","unstructured":"Kurtanovic Z, Maalej W (2018) On user rationale in software engineering. Requir Eng 23(3):357\u2013379. https:\/\/doi.org\/10.1007\/s00766-018-0293-2","journal-title":"Requir Eng"},{"key":"9960_CR22","doi-asserted-by":"publisher","unstructured":"Lin B, Zampetti F, Bavota G, Di Penta M, Lanza M, Oliveto R (2018) Sentiment analysis for software engineering: How far can we go?. In: 2018 IEEE\/ACM 40th International Conference on Software Engineering (ICSE), pp 94\u2013104. https:\/\/doi.org\/10.1145\/3180155.3180195","DOI":"10.1145\/3180155.3180195"},{"key":"9960_CR23","doi-asserted-by":"crossref","unstructured":"Lin B, Zampetti F, Bavota G, Penta MD, Lanza M (2019) Pattern-based Mining of Opinions in Q&A Websites. In: Proceedings of the 41st ACM\/IEEE international conference on software engineering ICSE","DOI":"10.1109\/ICSE.2019.00066"},{"key":"9960_CR24","doi-asserted-by":"publisher","unstructured":"M\u00e4ntyl\u00e4 M, Adams B, Destefanis G, Graziotin D, Ortu M (2016) Mining valence, arousal, and dominance: Possibilities for detecting burnout and productivity?. In: Proceedings of the 13th international conference on mining software repositories, ACM, New York, NY, USA, MSR \u201916, pp 247\u2013258. https:\/\/doi.org\/10.1145\/2901739.2901752","DOI":"10.1145\/2901739.2901752"},{"key":"9960_CR25","doi-asserted-by":"publisher","unstructured":"M\u00e4ntyl\u00e4 MV, Novielli N, Lanubile F, Claes M, Kuutila M (2017) Bootstrapping a lexicon for emotional arousal in software engineering. In: Proceedings of the 14th international conference on mining software repositories, IEEE Press, Piscataway, NJ, USA, MSR \u201917, pp 198\u2013202. https:\/\/doi.org\/10.1109\/MSR.2017.47","DOI":"10.1109\/MSR.2017.47"},{"issue":"1","key":"9960_CR26","doi-asserted-by":"publisher","first-page":"521","DOI":"10.1007\/s10664-017-9526-0","volume":"23","author":"A Murgia","year":"2018","unstructured":"Murgia A, Ortu M, Tourani P, Adams B, Demeyer S (2018) An exploratory qualitative and quantitative analysis of emotions in issue report comments of open source systems. Empir Softw Eng 23(1):521\u2013564. https:\/\/doi.org\/10.1007\/s10664-017-9526-0","journal-title":"Empir Softw Eng"},{"key":"9960_CR27","doi-asserted-by":"publisher","unstructured":"Novielli N, Calefato F, Lanubile F (2015) The challenges of sentiment detection in the social programmer ecosystem. In: Proceedings of the 7th International Workshop on Social Software Engineering, ACM, New York, NY, USA, SSE 2015, pp 33\u201340. https:\/\/doi.org\/10.1145\/2804381.2804387","DOI":"10.1145\/2804381.2804387"},{"key":"9960_CR28","doi-asserted-by":"publisher","unstructured":"Novielli N, Calefato F, Lanubile F (2018a) A gold standard for emotion annotation in stack overflow. In: Proceedings of the 15th international conference on mining software repositories, ACM, New York, NY, USA, MSR \u201918, pp 14\u201317. https:\/\/doi.org\/10.1145\/3196398.3196453","DOI":"10.1145\/3196398.3196453"},{"key":"9960_CR29","doi-asserted-by":"publisher","unstructured":"Novielli N, Girardi D, Lanubile F (2018b) A benchmark study on sentiment analysis for software engineering research. In: Proceedings of the 15th international conference on mining software repositories, ACM, New York, NY, USA, MSR \u201918, pp 364\u2013375. https:\/\/doi.org\/10.1145\/3196398.3196403","DOI":"10.1145\/3196398.3196403"},{"key":"9960_CR30","doi-asserted-by":"publisher","first-page":"180","DOI":"10.1016\/j.jss.2018.11.016","volume":"148","author":"N Novielli","year":"2019","unstructured":"Novielli N, Begel A, Maalej W (2019) Introduction to the special issue on affect awareness in software engineering. J Syst Softw 148:180\u2013182. https:\/\/doi.org\/10.1016\/j.jss.2018.11.016","journal-title":"J Syst Softw"},{"issue":"3","key":"9960_CR31","doi-asserted-by":"publisher","first-page":"86","DOI":"10.1109\/MS.2020.2968557","volume":"37","author":"N Novielli","year":"2020","unstructured":"Novielli N, Calefato F, Lanubile F (2020) Love, joy, anger, sadness, fear, and surprise: Se needs special kinds of ai: A case study on text mining and se. IEEE Softw 37(3):86\u201391","journal-title":"IEEE Softw"},{"key":"9960_CR32","doi-asserted-by":"publisher","unstructured":"Novielli N, Girardi D, Lanubile F (2020) Can we use se-specific sentiment analysis tools in a cross-platform setting?. In: Proceedings of the 17th international conference on mining software repositories, ACM, New York, NY, USA, MSR \u201920. https:\/\/doi.org\/10.1145\/3379597.3387446","DOI":"10.1145\/3379597.3387446"},{"key":"9960_CR33","doi-asserted-by":"crossref","unstructured":"Ortu M, Murgia A, Destefanis G, Tourani P, Tonelli R, Marchesi M, Adams B (2016) The emotional side of software developers in jira. In: Proceedings of the 13th international conference on mining software repositories. ACM, pp 480\u2013483","DOI":"10.1145\/2901739.2903505"},{"issue":"1-2","key":"9960_CR34","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1561\/1500000011","volume":"2","author":"B Pang","year":"2008","unstructured":"Pang B, Lee L (2008) Opinion mining and sentiment analysis. Found Trends Inf Retr 2(1-2):1\u2013135. https:\/\/doi.org\/10.1561\/1500000011","journal-title":"Found Trends Inf Retr"},{"key":"9960_CR35","doi-asserted-by":"publisher","unstructured":"Panichella S, Di Sorbo A, Guzman E, Visaggio CA, Canfora G, Gall HC (2015) How can i improve my app? classifying user reviews for software maintenance and evolution. In: Proceedings of the 2015 IEEE international conference on software maintenance and evolution (ICSME), IEEE Computer Society, Washington, DC, USA, ICSME \u201915, pp 281\u2013290. https:\/\/doi.org\/10.1109\/ICSM.2015.7332474","DOI":"10.1109\/ICSM.2015.7332474"},{"key":"9960_CR36","doi-asserted-by":"publisher","unstructured":"Pletea D, Vasilescu B, Serebrenik A (2014) Security and emotion: Sentiment analysis of security discussions on github. In: Proceedings of the 11th working conference on mining software repositories, ACM, New York, NY, USA, MSR 2014, pp 348\u2013351. https:\/\/doi.org\/10.1145\/2597073.2597117","DOI":"10.1145\/2597073.2597117"},{"issue":"2","key":"9960_CR37","doi-asserted-by":"publisher","first-page":"211","DOI":"10.1007\/s10664-008-9060-1","volume":"13","author":"FJ Shull","year":"2008","unstructured":"Shull FJ, Carver JC, Vegas S, Juristo N (2008) The role of replications in empirical software engineering. Empir Softw Eng 13(2):211\u2013218. https:\/\/doi.org\/10.1007\/s10664-008-9060-1","journal-title":"Empir Softw Eng"},{"key":"9960_CR38","doi-asserted-by":"publisher","unstructured":"Sinha V, Lazar A, Sharif B (2016) Analyzing developer sentiment in commit logs. In: Proceedings of the 13th international conference on mining software repositories, ACM, New York, NY, USA, MSR \u201916, pp 520\u2013523. https:\/\/doi.org\/10.1145\/2901739.2903501","DOI":"10.1145\/2901739.2903501"},{"key":"9960_CR39","doi-asserted-by":"crossref","unstructured":"Storey MA (2012) The evolution of the social programmer. In: Proceedings of the 9th IEEE working conference on mining software repositories, IEEE Press, MSR \u201912, p 140","DOI":"10.1109\/MSR.2012.6224273"},{"issue":"1","key":"9960_CR40","doi-asserted-by":"publisher","first-page":"163","DOI":"10.1002\/asi.21662","volume":"63","author":"M Thelwall","year":"2012","unstructured":"Thelwall M, Buckley K, Paltoglou G (2012) Sentiment strength detection for the social web. J Am Soc Inf Sci Technol 63(1):163\u2013173. https:\/\/doi.org\/10.1002\/asi.21662","journal-title":"J Am Soc Inf Sci Technol"},{"key":"9960_CR41","doi-asserted-by":"publisher","unstructured":"Uddin G, Khomh F (2017) Opiner: an opinion search and summarization engine for apis. In: Proceedings of the 32nd IEEE\/ACM International Conference on Automated Software Engineering, ASE 2017, Urbana, IL, USA, October 30 - November 03, 2017, pp 978\u2013983. https:\/\/doi.org\/10.1109\/ASE.2017.8115715","DOI":"10.1109\/ASE.2017.8115715"},{"issue":"5","key":"9960_CR42","first-page":"360","volume":"37","author":"AJ Viera","year":"2005","unstructured":"Viera AJ, Garrett JM (2005) Understanding interobserver agreement: the kappa statistic. Family Med 37(5):360\u20133","journal-title":"Family Med"},{"key":"9960_CR43","doi-asserted-by":"publisher","first-page":"49","DOI":"10.1007\/978-3-319-30319-2_3","volume-title":"The Comprehension of Figurative Language: What Is the Influence of Irony and Sarcasm on NLP Techniques?","author":"L Weitzel","year":"2016","unstructured":"Weitzel L, Prati RC, Aguiar RF (2016) The Comprehension of Figurative Language: What Is the Influence of Irony and Sarcasm on NLP Techniques?. Springer International Publishing, Cham, pp 49\u201374. https:\/\/doi.org\/10.1007\/978-3-319-30319-2_3"},{"key":"9960_CR44","doi-asserted-by":"publisher","unstructured":"Werner C, Tapuc G, Montgomery L, Sharma D, Dodos S, Damian D (2018) How angry are your customers? sentiment analysis of support tickets that escalate. In: 2018 1st International Workshop on Affective Computing for Requirements Engineering (AffectRE), pp 1\u20138. https:\/\/doi.org\/10.1109\/AffectRE.2018.00006","DOI":"10.1109\/AffectRE.2018.00006"},{"key":"9960_CR45","doi-asserted-by":"publisher","unstructured":"Zhang T, Xu B, Thung F, Haryono SA, Lo D, Jiang L (2020) Sentiment Analysis for Software Engineering: How Far Can Pre-trained Transformer Models Go?. In: Proceedings of the 36th IEEE international conference on software maintenance and evolution (ICSME 2020), IEEE, New York, NY, USA, ICSME \u201920, pp 70\u201380. https:\/\/doi.org\/10.1109\/ICSME46990.2020.00017","DOI":"10.1109\/ICSME46990.2020.00017"}],"container-title":["Empirical Software Engineering"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10664-021-09960-w.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10664-021-09960-w\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10664-021-09960-w.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,6,23]],"date-time":"2021-06-23T12:49:02Z","timestamp":1624452542000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10664-021-09960-w"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,6,5]]},"references-count":45,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2021,7]]}},"alternative-id":["9960"],"URL":"https:\/\/doi.org\/10.1007\/s10664-021-09960-w","relation":{},"ISSN":["1382-3256","1573-7616"],"issn-type":[{"type":"print","value":"1382-3256"},{"type":"electronic","value":"1573-7616"}],"subject":[],"published":{"date-parts":[[2021,6,5]]},"assertion":[{"value":"5 March 2021","order":1,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"5 June 2021","order":2,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}}],"article-number":"77"}}