{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,5]],"date-time":"2026-02-05T19:48:51Z","timestamp":1770320931106,"version":"3.49.0"},"reference-count":40,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2026,2,5]],"date-time":"2026-02-05T00:00:00Z","timestamp":1770249600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Artif. Intell."],"abstract":"<jats:sec>\n                    <jats:title>Context<\/jats:title>\n                    <jats:p>Fraud and corruption are among the main crimes affecting public institutions, with the healthcare sector being particularly vulnerable due to its structural complexity, the coexistence of public and private providers, the large number of actors involved, the globalized nature of supply chains, the high financial costs, and the information asymmetry among stakeholders. These factors weaken healthcare systems, resulting in resource waste, reduced resilience during medical emergencies, and limited access to essential services.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Objective<\/jats:title>\n                    <jats:p>This study aims to evaluate automatic text summarization methods by comparing the quality of machine-generated summaries with those produced by humans, from the perspective of Data Scientists and SUS Auditors, within the context of audits carried out by the National Department of Unified Health System (Sistema \u00danico de Sa\u00fade\u2014SUS) Auditing (AudSUS).<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Method<\/jats:title>\n                    <jats:p>A controlled experiment was conducted to assess the performance of Small Language Models (SLMs) in summarization tasks, using the metrics ROUGE-N, ROUGE-L, BLEU, METEOR, and BERTScore. In addition, the consistency of results across 35 runs, their contribution to reducing information overload, and their pairwise performances were evaluated.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>The models NousResearch\/Hermes-3-Llama-3.2-3B, Qwen\/Qwen2.5-7B-Instruct, and meta-llama\/Llama-3.2-3B-Instruct achieved the highest average performances across all metrics, standing out for their ability to preserve contextual meaning and synthesize essential information more effectively than human-generated summaries.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Conclusion<\/jats:title>\n                    <jats:p>The findings highlight the potential of SLMs as tools to reduce information overload, thereby enhancing the effectiveness of the analytical phase of audits and enabling faster preparation of teams for the operational stage.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.3389\/frai.2026.1708993","type":"journal-article","created":{"date-parts":[[2026,2,5]],"date-time":"2026-02-05T06:44:20Z","timestamp":1770273860000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["Small language models applied in text summarization task of health-related news to improve public health audit: an experimental case study"],"prefix":"10.3389","volume":"9","author":[{"given":"Alysson","family":"Guimar\u00e3es","sequence":"first","affiliation":[{"name":"Postgraduate Program in Computer Science (PROCC), Federal University of Sergipe","place":["S\u00e3o Crist\u00f3v\u00e3o, Brazil"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Methanias","family":"Cola\u00e7o Junior","sequence":"additional","affiliation":[{"name":"Postgraduate Program in Computer Science (PROCC), Federal University of Sergipe","place":["S\u00e3o Crist\u00f3v\u00e3o, Brazil"]},{"name":"Laboratory for Technological Innovation in Health (LAIS), Onofre Lopes University Hospital","place":["Federal University of Rio Grande do Norte (UFRN), Brazil"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Samuel Santana","family":"De Almeida","sequence":"additional","affiliation":[{"name":"Postgraduate Program in Computer Science (PROCC), Federal University of Sergipe","place":["S\u00e3o Crist\u00f3v\u00e3o, Brazil"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Gabriely","family":"Garcia Ferreira de Ara\u00fajo","sequence":"additional","affiliation":[{"name":"Department of Science and Technology, Federal University of Rio Grande do Norte","place":["Natal, Brazil"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Raphael Silva","family":"Fontes","sequence":"additional","affiliation":[{"name":"Center for Innovation and Advanced Technology (NAVI)","place":["Federal Institute of Rio Grande do Norte (IFRN), Brazil"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Helder","family":"Prado","sequence":"additional","affiliation":[{"name":"Postgraduate Program in Computer Science (PROCC), Federal University of Sergipe","place":["S\u00e3o Crist\u00f3v\u00e3o, Brazil"]},{"name":"Center for Innovation and Advanced Technology (NAVI)","place":["Federal Institute of Rio Grande do Norte (IFRN), Brazil"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Luca Pareja","family":"Credidio Freire Alves","sequence":"additional","affiliation":[{"name":"Department of Science and Technology, Federal University of Rio Grande do Norte","place":["Natal, Brazil"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Natan","family":"Matos","sequence":"additional","affiliation":[{"name":"Postgraduate Program in Computer Science (PROCC), Federal University of Sergipe","place":["S\u00e3o Crist\u00f3v\u00e3o, Brazil"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Ricardo Alexsandro","family":"de Medeiros Valentim","sequence":"additional","affiliation":[{"name":"Laboratory for Technological Innovation in Health (LAIS), Onofre Lopes University Hospital","place":["Federal University of Rio Grande do Norte (UFRN), Brazil"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jo\u00e3o Paulo Queiroz","family":"dos Santos","sequence":"additional","affiliation":[{"name":"Center for Innovation and Advanced Technology (NAVI)","place":["Federal Institute of Rio Grande do Norte (IFRN), Brazil"]}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1965","published-online":{"date-parts":[[2026,2,5]]},"reference":[{"key":"B1","doi-asserted-by":"publisher","first-page":"236","DOI":"10.1016\/j.asoc.2015.04.050","article-title":"An unsupervised approach to generating generic summaries of documents","volume":"34","author":"Alguliyev","year":"2015","journal-title":"Appl. Soft Comput. J"},{"key":"B2","doi-asserted-by":"publisher","first-page":"728","DOI":"10.1109\/TSE.1984.5010301","article-title":"A methodology for collecting valid software engineering data","volume":"10","author":"Basili","year":"1984","journal-title":"IEEE Trans. Softw. Eng"},{"key":"B3","author":"Benjelloun","year":"2015"},{"key":"B4","doi-asserted-by":"publisher","first-page":"101325","DOI":"10.1016\/j.patter.2025.101325","article-title":"Tucano: advancing neural text generation for Portuguese","volume":"6","author":"Corr\u00eaa","year":"2025","journal-title":"Patterns"},{"key":"B5","doi-asserted-by":"publisher","DOI":"10.25286\/repa.v5i1.1179","article-title":"Aloca\u00e7\u00e3 de t\u00f3picos latentes \u2014 um modelo para segmenta\u00e7\u00e3o de dados de auditoria do governo de pe","author":"do Amaral","year":"2020","journal-title":"Rev. Eng. Pesqui. Apl. 5"},{"key":"B6","doi-asserted-by":"publisher","first-page":"113679","DOI":"10.1016\/j.eswa.2020.113679","article-title":"Automatic text summarization: a comprehensive survey","volume":"165","author":"El-Kassas","year":"2021","journal-title":"Expert Syst. Appl"},{"key":"B7","doi-asserted-by":"publisher","first-page":"457","DOI":"10.1613\/jair.1523","article-title":"Lexrank: graph-based lexical centrality as salience in text summarization","volume":"22","author":"Erkan","year":"2004","journal-title":"J. Artif. Intell. Res"},{"key":"B8","doi-asserted-by":"publisher","first-page":"6518","DOI":"10.1109\/ACCESS.2024.3349952","article-title":"A survey of text classification with transformers: How wide? How large? How long? How accurate? How expensive? How safe?","volume":"12","author":"Fields","year":"2024","journal-title":"IEEE Access"},{"key":"B9","doi-asserted-by":"publisher","author":"Fontes","year":"2023","DOI":"10.5753\/sbcas_estendido.2023.231515"},{"key":"B10","doi-asserted-by":"publisher","DOI":"10.34740\/KAGGLE\/M\/3301","author":"Gemma Team","year":"2024","journal-title":"Gemma"},{"key":"B11","doi-asserted-by":"publisher","author":"Haghighi","year":"2009","DOI":"10.3115\/1620754.1620807"},{"key":"B12","doi-asserted-by":"publisher","first-page":"12346","DOI":"10.3390\/app132212346","article-title":"Natural language processing adoption in governments and future research directions: a systematic review","volume":"13","author":"Jiang","year":"2023","journal-title":"Appl. Sci"},{"key":"B13","doi-asserted-by":"publisher","first-page":"159","DOI":"10.2307\/2529310","article-title":"The measurement of observer agreement for categorical data","volume":"33","author":"Landis","year":"1977","journal-title":"Biometrics"},{"key":"B14","doi-asserted-by":"publisher","first-page":"228","DOI":"10.3115\/1626355.1626389","author":"Lavie","year":"2007","journal-title":"Proceedings of the Second Workshop on Statistical Machine Translation"},{"key":"B15","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv:1910.13461","article-title":"BART: denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension","author":"Lewis","year":"2019","journal-title":"arXiv [preprint]"},{"key":"B16","first-page":"74","author":"Lin","year":"2004"},{"key":"B17","doi-asserted-by":"publisher","first-page":"1865","DOI":"10.1093\/jamia\/ocae037","article-title":"Taiyi: A bilingual fine-tuned large language model for diverse biomedical tasks","volume":"31","author":"Luo","year":"2023","journal-title":"J. Am. Med. Inform. Assoc"},{"key":"B18","doi-asserted-by":"publisher","first-page":"634","DOI":"10.2471\/BLT.18.209502","article-title":"The sustainable development goals as a framework to combat health-sector corruption","volume":"96","author":"Mackey","year":"2018","journal-title":"Bull. World Health Organ"},{"key":"B19","doi-asserted-by":"publisher","first-page":"121086","DOI":"10.1016\/j.techfore.2021.121086","article-title":"Competitive intelligence: a unified view and modular definition","volume":"173","author":"Madureira","year":"2021","journal-title":"Technol. Forecast. Soc. Change"},{"key":"B20","doi-asserted-by":"publisher","first-page":"103","DOI":"10.1561\/1500000015","article-title":"Automatic summarization","volume":"5","author":"Nenkova","year":"2011","journal-title":"Found. Trends Inf. Retr"},{"key":"B21","volume-title":"Proceedings of the Document Understanding Conference","year":"2025"},{"key":"B22","doi-asserted-by":"publisher","first-page":"311","DOI":"10.3115\/1073083.1073135","author":"Papineni","year":"2002","journal-title":"Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics"},{"key":"B23","first-page":"610","author":"Paula","year":"2024","journal-title":"Proceedings of the 16th International Conference on Computational Processing of Portuguese, Vol. 1"},{"key":"B24","doi-asserted-by":"publisher","first-page":"226","DOI":"10.1007\/978-3-031-45392-2_15","author":"Pires","year":"2023","journal-title":"Intelligent Systems"},{"key":"B25","doi-asserted-by":"publisher","first-page":"13755","DOI":"10.1038\/s41598-025-98483-1","article-title":"Industrial applications of large language models","volume":"15","author":"Raza","year":"2025","journal-title":"Sci. Rep"},{"key":"B26","doi-asserted-by":"publisher","first-page":"55","DOI":"10.54254\/2755-2721\/97\/20241406","article-title":"Advancements and applications of large language models in natural language processing: a comprehensive review","volume":"97","author":"Ren","year":"2024","journal-title":"Appl. Comput. Eng"},{"key":"B27","doi-asserted-by":"publisher","first-page":"45","DOI":"10.1016\/j.knosys.2018.10.021","article-title":"Extractive single document summarization using multi-objective optimization: exploring self-organized differential evolution, grey wolf optimizer and water cycle algorithm","volume":"164","author":"Saini","year":"2019","journal-title":"Knowl. Based Syst"},{"key":"B28","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1016\/j.knosys.2017.11.029","article-title":"Extractive multi-document text summarization using a multi-objective artificial bee colony optimization approach","volume":"159","author":"Sanchez-Gomez","year":"2018","journal-title":"Knowl. Based Syst"},{"key":"B29","doi-asserted-by":"publisher","first-page":"106231","DOI":"10.1016\/j.asoc.2020.106231","article-title":"A decomposition-based multi-objective optimization approach for extractive multi-document text summarization","volume":"91","author":"Sanchez-Gomez","year":"2020","journal-title":"Appl. Soft Comput. J"},{"key":"B30","doi-asserted-by":"publisher","first-page":"116769","DOI":"10.1016\/j.eswa.2022.116769","article-title":"A multi-objective memetic algorithm for query-oriented text summarization: medicine texts as a case study","volume":"198","author":"Sanchez-Gomez","year":"2022","journal-title":"Expert Syst. Appl"},{"key":"B31","doi-asserted-by":"publisher","first-page":"101721","DOI":"10.1016\/j.swevo.2024.101721","article-title":"An indicator-based multi-objective variable neighborhood search approach for query-focused summarization","volume":"91","author":"Sanchez-Gomez","year":"2024","journal-title":"Swarm Evol. Comput"},{"key":"B32","doi-asserted-by":"publisher","author":"Steinberger","year":"2004","DOI":"10.1007\/978-3-540-30198-1_25"},{"key":"B33","unstructured":"Team\n              M.\n            \n          \n          Llama 3.2: Revolutionizing Edge AI and Vision with Open, Customizable Models"},{"key":"B34","unstructured":"Team\n              Q.\n            \n          \n          Qwen2.5: A Party of Foundation Models"},{"key":"B35","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv:2408.11857","article-title":"Hermes 3 technical report","author":"Teknium","year":"2024","journal-title":"arXiv [preprint]"},{"key":"B36","volume-title":"Introdu\u00e7\u00e3o \u00e0 Engenharia de Software","author":"Travassos","year":"2020"},{"key":"B37","doi-asserted-by":"publisher","first-page":"1134","DOI":"10.1038\/s41591-024-02855-5","article-title":"Adapted large language models can outperform medical experts in clinical text summarization","volume":"30","author":"Van Veen","year":"2024","journal-title":"Nat. Med"},{"key":"B38","doi-asserted-by":"publisher","first-page":"6173","DOI":"10.1145\/3711896.3736563","author":"Wang","year":"2025"},{"key":"B39","author":"Woodsend","year":"2011"},{"key":"B40","unstructured":"\u201cBERTScore: evaluating text generation with BERT,\u201d\n          \n          \n            \n              Zhang\n              T.\n            \n            \n              Kishore\n              V.\n            \n            \n              Wu\n              F.\n            \n            \n              Weinberger\n              K. Q.\n            \n            \n              Artzi\n              Y.\n            \n          \n          Proceedings of the International Conference on Learning Representations (ICLR)\n          \n          2020"}],"container-title":["Frontiers in Artificial Intelligence"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/frai.2026.1708993\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,2,5]],"date-time":"2026-02-05T06:44:23Z","timestamp":1770273863000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/frai.2026.1708993\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,2,5]]},"references-count":40,"alternative-id":["10.3389\/frai.2026.1708993"],"URL":"https:\/\/doi.org\/10.3389\/frai.2026.1708993","relation":{},"ISSN":["2624-8212"],"issn-type":[{"value":"2624-8212","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,2,5]]},"article-number":"1708993"}}