{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,30]],"date-time":"2026-04-30T23:04:08Z","timestamp":1777590248482,"version":"3.51.4"},"reference-count":30,"publisher":"MDPI AG","issue":"1","license":[{"start":{"date-parts":[[2025,3,6]],"date-time":"2025-03-06T00:00:00Z","timestamp":1741219200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"CONACYT","award":["1078852"],"award-info":[{"award-number":["1078852"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Publications"],"abstract":"<jats:p>Preliminary activities of searching and selecting relevant articles are crucial in scientific research to determine the state of the art (SOTA) and enhance overall outcomes. While there are automatic tools for keyword extraction, these algorithms are often computationally expensive, storage-intensive, and reliant on institutional subscriptions for metadata retrieval. Most importantly, they still require manual selection of literature. This paper introduces a framework that automates keyword searching in article abstracts to help select relevant literature for the SOTA by identifying key terms matching that we, hereafter, call source words. A case study in the food and beverage industry is provided to demonstrate the algorithm\u2019s application. In the study, five relevant knowledge areas were defined to guide literature selection. The database from scientific repositories was categorized using six classification rules based on impact factor (IF), Open Access (OA) status, and JCR journal ranking. This classification revealed the knowledge area with the highest presence and highlighted the effectiveness of the selection rules in identifying articles for the SOTA. The approach included a panel of experts who confirmed the algorithm\u2019s effectiveness in identifying source words in high-quality articles. The algorithm\u2019s performance was evaluated using the F1 Score, which reached 0.83 after filtering out non-relevant articles. This result validates the algorithm\u2019s ability to extract significant source words and demonstrates its usefulness in building the SOTA by focusing on the most scientifically impactful articles.<\/jats:p>","DOI":"10.3390\/publications13010011","type":"journal-article","created":{"date-parts":[[2025,3,6]],"date-time":"2025-03-06T06:02:13Z","timestamp":1741240933000},"page":"11","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":1,"title":["Automation Applied to the Collection and Generation of Scientific Literature"],"prefix":"10.3390","volume":"13","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-6950-8134","authenticated-orcid":false,"given":"Nadia Paola","family":"Valadez-de la Paz","sequence":"first","affiliation":[{"name":"Departamento de Ingenieria Industrial, Tecnologico Nacional de Mexico, Instituto Tecnologico de Celaya, Celaya 38010, Mexico"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1687-2140","authenticated-orcid":false,"given":"Jose Antonio","family":"Vazquez-Lopez","sequence":"additional","affiliation":[{"name":"Departamento de Ingenieria Industrial, Tecnologico Nacional de Mexico, Instituto Tecnologico de Celaya, Celaya 38010, Mexico"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0488-7799","authenticated-orcid":false,"given":"Aidee","family":"Hernandez-Lopez","sequence":"additional","affiliation":[{"name":"Departamento de Ingenieria Industrial, Universidad del SABES, Campus Celaya, Leon 37234, Mexico"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4992-2992","authenticated-orcid":false,"given":"Jaime Francisco","family":"Aviles-Vi\u00f1as","sequence":"additional","affiliation":[{"name":"Mechatronics Group, Engineering Faculty, Autonomous University of Yucatan, Merida 97302, Mexico"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5742-3096","authenticated-orcid":false,"given":"Jose Luis","family":"Navarro-Gonzalez","sequence":"additional","affiliation":[{"name":"Departamento de Ingenieria Mecanica, Tecnologico Nacional de Mexico, Instituto Tecnologico de Saltillo, Saltillo 25280, Mexico"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2022-8048","authenticated-orcid":false,"given":"Alfredo Valentin","family":"Reyes-Acosta","sequence":"additional","affiliation":[{"name":"Facultad de Sistemas, Unidad Saltillo, Universidad Autonoma de Coahuila, Saltillo 25350, Mexico"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6405-5519","authenticated-orcid":false,"given":"Ismael","family":"Lopez-Juarez","sequence":"additional","affiliation":[{"name":"Mechatronics Group, Engineering Faculty, Autonomous University of Yucatan, Merida 97302, Mexico"},{"name":"Robotics and Advanced Manufacturing Department, Centre for Research and Advanced Studies (CINVESTAV), Ramos Arizpe 25900, Mexico"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2025,3,6]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Abdin, A. Y., and De Pretis, F. (2024). Measuring open access uptake: Methods and metrics to assess a market transformation. Publications, 12, (4).","DOI":"10.3390\/publications12040033"},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Ali, A., Tufail, A., De Silva, L. C., and Abas, P. E. (2024). Innovating patent retrieval: A comprehensive review of techniques, trends, and challenges in prior art searches. Applied System Innovation, 7, (5).","DOI":"10.3390\/asi7050091"},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Alkhamissi, B., Verma, S., Yu, P., Jin, Z., Asli, C., and Diab, M. (2023). OPT-R: Exploring the role of explanations in finetuning and prompting for reasoning skills of large language models. Proceedings of the 1st workshop on natural language reasoning and structured explanations (NLRSE), Association for Computational Linguistics.","DOI":"10.18653\/v1\/2023.nlrse-1.10"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Alshami, A., Elsayed, M., Ali, E., Eltoukhy, A. E. E., and Zayed, T. (2023). Harnessing the power of ChatGPT for automating systematic review process: Methodology, case study, limitations, and future directions. Systems, 11, (7).","DOI":"10.3390\/systems11070351"},{"key":"ref_5","unstructured":"arXiv API Access (2025, February 02). arXiv API access, Available online: https:\/\/info.arxiv.org\/help\/api\/index.html."},{"key":"ref_6","unstructured":"Asociaci\u00f3n Espa\u00f1ola de Normalizaci\u00f3n y Certificaci\u00f3n (1991). Information and documentation\u2014Criteria for the assessment of information retrieval systems (UNE 50-121-91), Asociaci\u00f3n Espa\u00f1ola de Normalizaci\u00f3n y Certificaci\u00f3n."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"37","DOI":"10.3233\/ISU-1984-41-204","article-title":"Optimizing convenient on-line access to bibliographic databases","volume":"4","author":"Cleverdon","year":"1984","journal-title":"Information Services & Use"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"2171","DOI":"10.1007\/s00607-023-01181-x","article-title":"Artificial intelligence to automate the systematic review of scientific literature","volume":"105","author":"Romero","year":"2023","journal-title":"Computing"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Golinelli, D., Nuzzolese, A. G., Sanmarchi, F., Bulla, L., Mongiov\u00ec, M., Gangemi, A., and Rucci, P. (2022). Semi-automatic systematic literature reviews and information extraction of COVID-19 scientific evidence: Description and preliminary results of the COKE project. Information, 13, (3).","DOI":"10.3390\/info13030117"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"204","DOI":"10.1016\/j.cie.2012.03.002","article-title":"A survey of control-chart pattern-recognition literature (1991\u20132010) based on a new conceptual classification scheme","volume":"63","author":"Hachicha","year":"2012","journal-title":"Computers & Industrial Engineering"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"14","DOI":"10.1038\/d41586-023-03290-1","article-title":"Japanese research is no longer world class\u2014Here\u2019s why","volume":"623","author":"Ikarashi","year":"2023","journal-title":"Nature"},{"key":"ref_12","unstructured":"International Organization for Standardization (1985). Information and documentation\u2014Methods for assessing the quality of information retrieval systems (ISO 5963:1985), International Organization for Standardization."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"195370","DOI":"10.1109\/ACCESS.2024.3514916","article-title":"FaN-REMs: Fair and normalized retrieval evaluation metrics for learning retrieval systems","volume":"12","author":"Jaiswal","year":"2024","journal-title":"IEEE Access"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"100134","DOI":"10.1016\/j.slast.2024.100134","article-title":"ProtoCode: Leveraging large language models (LLMs) for automated generation of machine-readable PCR protocols from scientific publications","volume":"29","author":"Jiang","year":"2024","journal-title":"SLAS Technology"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"51740","DOI":"10.1109\/ACCESS.2024.3377239","article-title":"Reliable information retrieval systems performance evaluation: A review","volume":"12","author":"Joseph","year":"2024","journal-title":"IEEE Access"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Kirby, A. (2023). Exploratory bibliometrics: Using VOSviewer as a preliminary research tool. Publications, 11, (1).","DOI":"10.3390\/publications11010010"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"195","DOI":"10.1145\/3560815","article-title":"Pre-train, prompt, and predict: A systematic survey of prompting methods in natural language processing","volume":"55","author":"Liu","year":"2023","journal-title":"ACM Computing Surveys"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Manning, C. D., Raghavan, P., and Sch\u00fctze, H. (2008). Introduction to information retrieval, Cambridge University Press.","DOI":"10.1017\/CBO9780511809071"},{"key":"ref_19","unstructured":"Miao, F., and Holmes, W. (2023). Guidance for generative ai in education and research, UNESCO."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Morales-Alarc\u00f3n, C. H., Bodero-Poveda, E., Villa-Y\u00e1nez, H. M., and Bu\u00f1ay-Guis\u00f1an, P. A. (2024). Blockchain and its application in the peer review of scientific works: A systematic review. Publications, 12, (4).","DOI":"10.3390\/publications12040040"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"e2401238121","DOI":"10.1073\/pnas.2401238121","article-title":"Automating the practice of science\u2014Opportunities, challenges, and implications","volume":"122","author":"Musslick","year":"2025","journal-title":"Proceedings of the National Academy of Sciences USA"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"200","DOI":"10.1007\/s10462-024-10844-w","article-title":"Towards the automation of systematic reviews using natural language processing, machine learning, and deep learning: A comprehensive review","volume":"57","author":"Wiratunga","year":"2024","journal-title":"Artificial Intelligence Review"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"187","DOI":"10.1186\/s13643-023-02351-w","article-title":"Evaluation of a prototype machine learning tool to semi-automate data extraction for systematic literature reviews","volume":"12","author":"Panayi","year":"2023","journal-title":"Systematic Reviews"},{"key":"ref_24","unstructured":"Sarthi, P., Abdullah, S., Tuli, A., Khanna, S., Goldie, A., and Manning, C. D. (, January May). RAPTOR: Recursive abstractive processing for tree-organized retrieval. The Twelfth International Conference on Learning Representations, Vienna, Austria."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Schick, T., and Sch\u00fctze, H. (2021). It\u2019s not just size that matters: Small language models are also few-shot learners. Proceedings of the 2021 conference of the north american chapter of the association for computational linguistics: Human language technologies, Association for Computational Linguistics.","DOI":"10.18653\/v1\/2021.naacl-main.185"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"e55318","DOI":"10.2196\/55318","article-title":"An empirical evaluation of prompting strategies for large language models in zero-shot clinical natural language processing: Algorithm development and validation study","volume":"12","author":"Sivarajkumar","year":"2024","journal-title":"JMIR Medical Informatics"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Turgel, I. D., and Chernova, O. A. (2024). Open science alternatives to scopus and the web of science: A case study in regional resilience. Publications, 12, (4).","DOI":"10.3390\/publications12040043"},{"key":"ref_28","unstructured":"Vatsal, S., and Dubey, H. (2024). A survey of prompt engineering methods in large language models for different NLP tasks. arXiv."},{"key":"ref_29","unstructured":"Wu, S., Zhao, X., Yu, T., Zhang, R., Shen, C., Liu, H., Li, F., Zhu, H., Luo, J., Xu, L., and Zhang, X. (2021). Yuan 1.0: Large-scale pre-trained language model in zero-shot and few-shot learning. arXiv."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Zamiri, M., and Esmaeili, A. (2024). Methods and technologies for supporting knowledge sharing within learning communities: A systematic literature review. Administrative Sciences, 14, (1).","DOI":"10.3390\/admsci14010017"}],"container-title":["Publications"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2304-6775\/13\/1\/11\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,9]],"date-time":"2025-10-09T16:48:16Z","timestamp":1760028496000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2304-6775\/13\/1\/11"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,3,6]]},"references-count":30,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2025,3]]}},"alternative-id":["publications13010011"],"URL":"https:\/\/doi.org\/10.3390\/publications13010011","relation":{},"ISSN":["2304-6775"],"issn-type":[{"value":"2304-6775","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,3,6]]}}}