{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,31]],"date-time":"2026-08-31T10:16:52Z","timestamp":1788171412151,"version":"build-2803163510"},"reference-count":23,"publisher":"MDPI AG","issue":"5","license":[{"start":{"date-parts":[[2026,8,31]],"date-time":"2026-08-31T00:00:00Z","timestamp":1788134400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Med. Sci.","Medical Sciences"],"abstract":"<jats:p>Background\/Objectives: Artificial intelligence tools have emerged as promising methodological support for systematic reviews and health technology assessment (HTA). Smart infusion pump interoperability represents a relevant case study due to its implications for medication safety, nursing workflow, and hospital quality improvement. The aim was to evaluate the performance of artificial intelligence as a methodological support tool across a systematic review, using the evidence synthesis on smart infusion pump\u2013electronic health record interoperability as a case study. Methods: A systematic review following PRISMA 2020 guidelines was conducted. Searches were performed in MEDLINE, Embase, and Cochrane Library databases. AI-assisted tools (ChatGPT GPT-4 and Open Science Reviewer) were incorporated into data extraction, reporting appraisal based on STROBE criteria, and exploratory identification of methodological limitations under strict human supervision. Concordance between AI-assisted and manual extraction was evaluated descriptively. Results: Overall concordance between AI-assisted and manual data extraction was 82.5% (99\/120) across assessed variables. Agreement was highest for structured variables, including study design (10\/10; 100.0%), study identification variables (19\/20; 95.0%), and participant characteristics (36\/40; 90.0%). Agreement was lower for study content variables (18\/30; 60.0%) and methodological appraisal (16\/20; 80.0%). Among the 21 discrepancies, misclassification errors were most common (13\/21; 61.9%), followed by omissions (3\/21; 14.3%), incomplete data (3\/21; 14.3%), and hallucinations (2\/21; 9.5%). AI-assisted identification of methodological limitations showed substantial descriptive agreement with human assessments but demonstrated limited capacity for judgmental interpretations. Conclusions: Artificial intelligence demonstrated utility for structured review tasks such as data extraction and reporting appraisal, but showed limitations in tasks requiring interpretative and methodological judgement. Human oversight therefore remains essential throughout the review process. These findings derive from a single case study and should not be generalised beyond the evaluated context and AI tools.<\/jats:p>","DOI":"10.3390\/medsci14050533","type":"journal-article","created":{"date-parts":[[2026,8,31]],"date-time":"2026-08-31T09:21:19Z","timestamp":1788168079000},"page":"533","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Artificial Intelligence-Supported Evidence Synthesis: A Case Study of Smart Infusion Pump Interoperability"],"prefix":"10.3390","volume":"14","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-5420-7347","authenticated-orcid":false,"given":"Carlos","family":"Sanchez-Piedra","sequence":"first","affiliation":[{"name":"Agency for Health Technology Assessment, Instituto de Salud Carlos III, 28029 Madrid, Spain"},{"name":"Research Network on Chronicity, Primary Care and Health Promotion (RICAPPS), 20809 Madrid, Spain"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8252-2552","authenticated-orcid":false,"given":"Ivo","family":"Heyerdahl-Viau","sequence":"additional","affiliation":[{"name":"Orphan Drugs Laboratory, Department of Biological Systems, Universidad Aut\u00f3noma Metropolitana Unidad Xochimilco, Mexico City 04960, Mexico"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Esther-Elena","family":"Garcia-Carpintero","sequence":"additional","affiliation":[{"name":"Agency for Health Technology Assessment, Instituto de Salud Carlos III, 28029 Madrid, Spain"},{"name":"Research Network on Chronicity, Primary Care and Health Promotion (RICAPPS), 20809 Madrid, Spain"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6316-5054","authenticated-orcid":false,"given":"Juan-Manuel","family":"Martinez-Nu\u00f1ez","sequence":"additional","affiliation":[{"name":"Orphan Drugs Laboratory, Department of Biological Systems, Universidad Aut\u00f3noma Metropolitana Unidad Xochimilco, Mexico City 04960, Mexico"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1474-5228","authenticated-orcid":false,"given":"Francisco-Javier","family":"Prado-Galbarro","sequence":"additional","affiliation":[{"name":"Research Department, Hospital Infantil de M\u00e9xico Federico G\u00f3mez, Mexico City 06720, Mexico"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2026,8,31]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"1897","DOI":"10.2146\/ajhp120767","article-title":"Impact of Implementing Smart Infusion Pumps in a Pediatric Intensive Care Unit","volume":"70","year":"2013","journal-title":"Am. J. Health-Syst. Pharm."},{"key":"ref_2","first-page":"1089","article-title":"Evaluation of Intravenous Medication Errors with Smart Infusion Pumps in an Academic Medical Center","volume":"2013","author":"Ohashi","year":"2013","journal-title":"AMIA Annu. Symp. Proc."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"215","DOI":"10.1016\/j.cnc.2018.02.004","article-title":"Intravenous Smart Pumps: Usability Issues, Intravenous Medication Administration Error, and Patient Safety","volume":"30","author":"Giuliano","year":"2018","journal-title":"Crit. Care Nurs. Clin. N. Am."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"80","DOI":"10.1136\/qshc.2004.011957","article-title":"Insights from the Sharp End of Intravenous Medication Errors: Implications for Infusion Pump Technology","volume":"14","author":"Husch","year":"2005","journal-title":"Qual. Saf. Health Care"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"3707","DOI":"10.1111\/jan.14855","article-title":"Artificial Intelligence in Nursing: Priorities and Opportunities from an International Invitational Think-Tank of the Nursing and Artificial Intelligence Leadership Collaborative","volume":"77","author":"Ronquillo","year":"2021","journal-title":"J. Adv. Nurs."},{"key":"ref_6","first-page":"165","article-title":"Big Data and Nursing: Implications for the Future","volume":"232","author":"Topaz","year":"2017","journal-title":"Stud. Health Technol. Inform."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"77","DOI":"10.1186\/s13643-018-0740-7","article-title":"Making Progress with the Automation of Systematic Reviews: Principles of the International Collaboration for the Automation of Systematic Reviews (ICASR)","volume":"7","author":"Beller","year":"2018","journal-title":"Syst. Rev."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"n71","DOI":"10.1136\/bmj.n71","article-title":"The PRISMA 2020 Statement: An Updated Guideline for Reporting Systematic Reviews","volume":"372","author":"Page","year":"2021","journal-title":"BMJ"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"59","DOI":"10.1186\/s13643-015-0031-5","article-title":"Erratum to: Using text mining for study identification in systematic reviews: A systematic review of current approaches","volume":"4","author":"Thomas","year":"2015","journal-title":"Syst. Rev."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"193","DOI":"10.1093\/jamia\/ocv044","article-title":"RobotReviewer: Evaluation of a System for Automatically Assessing Bias in Clinical Trials","volume":"23","author":"Marshall","year":"2016","journal-title":"J. Am. Med. Inform. Assoc. JAMIA"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Wallace, B.C., Small, K., Brodley, C.E., and Trikalinos, T.A. (2010). Active Learning for Biomedical Citation Screening. Proceedings of the 16th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Association for Computing Machinery (ACM).","DOI":"10.1145\/1835804.1835829"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"125","DOI":"10.1038\/s42256-020-00287-7","article-title":"An Open Source Machine Learning Framework for Efficient and Transparent Systematic Reviews","volume":"3","author":"Schram","year":"2021","journal-title":"Nat. Mach. Intell."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Booth, A., Sutton, A., Clowes, M., and Martyn-St James, M. (2022). Systematic Approaches to a Successful Literature Review, SAGE Publications Ltd.","DOI":"10.4135\/9781036232009"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"230","DOI":"10.1136\/svn-2017-000101","article-title":"Artificial Intelligence in Healthcare: Past, Present and Future","volume":"2","author":"Jiang","year":"2017","journal-title":"Stroke Vasc. Neurol."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"1763","DOI":"10.7326\/ANNALS-25-00739","article-title":"Artificial Intelligence-Assisted Data Extraction with a Large Language Model: A Study Within Reviews","volume":"178","author":"Gartlehner","year":"2025","journal-title":"Ann. Intern. Med."},{"key":"ref_16","unstructured":"Schmidt, L., Hair, K., Graziosi, S., Campbell, F., Kapp, C., Khanteymoori, A., Craig, D., Engelbert, M., and Thomas, J. (2024). Exploring the Use of a Large Language Model for Data Extraction in Systematic Reviews: A Rapid Feasibility Study. arXiv."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Anh-Hoang, D., Tran, V., and Nguyen, L.M. (2025). Survey and Analysis of Hallucinations in Large Language Models: Attribution to Prompting Strategies or Model Behavior. Front. Artif. Intell., 8.","DOI":"10.3389\/frai.2025.1622292"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"6799","DOI":"10.1007\/s00146-025-02406-7","article-title":"Can Generative AI Reliably Synthesise Literature? Exploring Hallucination Issues in ChatGPT","volume":"40","author":"Adel","year":"2025","journal-title":"AI Soc."},{"key":"ref_19","unstructured":"Long, X., Boscardin, C., Maggio, L.A., Costello, J.A., Gonzales, R., Hammoudeh, R., Lai, K., Park, Y.S., and Gin, B.C. (2025). Hallucination vs Interpretation: Rethinking Accuracy and Precision in AI-Assisted Data Extraction for Knowledge Synthesis. arXiv."},{"key":"ref_20","unstructured":"Schroeder, N.L., Jaldi, C.D., and Zhang, S. (2025). Large Language Models with Human-In-The-Loop Validation for Systematic Review Data Extraction. arXiv."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"394","DOI":"10.1136\/bmjebm-2023-112597","article-title":"Integrating Large Language Models in Systematic Reviews: A Framework and Case Study Using ROBINS-I for Risk of Bias Assessment","volume":"29","author":"Hasan","year":"2024","journal-title":"BMJ Evid.-Based Med."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"82","DOI":"10.1186\/s13643-025-02842-y","article-title":"The Landscape of Artificial Intelligence Tools and Platforms for Evidence Synthesis: A Scoping Review","volume":"15","author":"Sousa","year":"2026","journal-title":"Syst. Rev."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Mahmoud, S.S., Demir, G.B., Oami, T., Okada, Y., Maeda, K., and Nakada, T.-A. (2026). Performance of Large Language Models and Prompt Engineering Strategies for Data Extraction in Systematic Reviews. Front. Digit. Health, 8.","DOI":"10.3389\/fdgth.2026.1799623"}],"container-title":["Medical Sciences"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2076-3271\/14\/5\/533\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,8,31]],"date-time":"2026-08-31T09:35:40Z","timestamp":1788168940000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2076-3271\/14\/5\/533"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,8,31]]},"references-count":23,"journal-issue":{"issue":"5","published-online":{"date-parts":[[2026,9]]}},"alternative-id":["medsci14050533"],"URL":"https:\/\/doi.org\/10.3390\/medsci14050533","relation":{},"ISSN":["2076-3271"],"issn-type":[{"value":"2076-3271","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,8,31]]}}}