{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,1]],"date-time":"2026-08-01T00:45:49Z","timestamp":1785545149575,"version":"3.56.0"},"reference-count":33,"publisher":"JMIR Publications Inc.","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["JMIR Med Inform"],"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:sec sec-type=\"background\">\n                    <jats:title>Background<\/jats:title>\n                    <jats:p>Evidence-based medicine combines scientific research, clinical expertise, and patient preferences to enhance the patient outcomes and improve health care quality. Clinical data are crucial in aligning medical decisions with evidence-based practices, whether derived from systematic research or real-world data sources. Quality assurance of clinical data, mainly through predictive quality algorithms and machine learning, is essential to mitigate risks such as misdiagnosis, inappropriate treatment, bias, and compromised patient safety. Furthermore, excellent quality of clinical data is a prerequisite for the replication of research results in order to gain insights from practice and real-world evidence.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec sec-type=\"objective\">\n                    <jats:title>Objective<\/jats:title>\n                    <jats:p>This study aims to demonstrate the varying quality of medical data in primary clinical source systems at a maximum care university hospital and provide researchers with insights into data reliability through predictive quality algorithms using machine learning techniques.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec sec-type=\"methods\">\n                    <jats:title>Methods<\/jats:title>\n                    <jats:p>A literature review was conducted to evaluate existing approaches to automated quality prediction. In addition, embedded in the process of integrating care data into a medical data integration center (MeDIC), metadata relevant to this clinical data was stored, considering factors such as data granularity and quality metrics. Completed patient cases with echocardiographic and laboratory findings as well as medication histories were selected from 2001 to 2023. Two authors manually reviewed the datasets and assigned a quality score for each entry, with 0 indicating unsatisfactory and 1 satisfactory quality. Since quality control was considered a binary problem, corresponding classifiers were used for the quality prediction. Logistic regression, k-nearest neighbors, a naive bayes classifier, a decision tree classifier, a random forest classifier, extreme gradient boosting (XGB), and support vector machines (SVM) were selected as machine learning algorithms. Based on preprocessing the dataset, training machine learning algorithms on echocardiographic, laboratory, and medication data, and assessing various prediction models, the most effective algorithms for quality classification were to be identified. The performance of the predictive quality algorithms was assessed based on accuracy, precision, recall, and scoring.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec sec-type=\"results\">\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>There were 450 patient cases with complete information extracted from the MeDIC data pool. The laboratory and medication datasets had to be limited to 4000 data entries each to enable manual review; the echocardiographic datasets comprised 750 examinations. XGB demonstrated the highest performance for the echocardiographic dataset with an area under the receiver operating characteristic curve (AUC-ROC) of 84.6%. For laboratory data, SVM achieved an AUC-ROC score of 89.8%, demonstrating superior discrimination performance. Finally, regarding the medication dataset, SVM showed the most balanced performance, achieving an AUC-ROC of 65.1%, the highest of all tested models.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec sec-type=\"conclusions\">\n                    <jats:title>Conclusions<\/jats:title>\n                    <jats:p>This proposal presents a template for predicting data quality and incorporating the resulting quality information into the metadata of a data integration center, a concept not previously implemented. The model was deployed for data inspection using a hybrid approach that combines the trained model with conventional inspection methods.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.2196\/60204","type":"journal-article","created":{"date-parts":[[2025,6,30]],"date-time":"2025-06-30T16:21:14Z","timestamp":1751300474000},"page":"e60204-e60204","source":"Crossref","is-referenced-by-count":6,"title":["Proposal for Using AI to Assess Clinical Data Integrity and Generate Metadata: Algorithm Development and Validation"],"prefix":"10.2196","volume":"13","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-7169-6090","authenticated-orcid":false,"given":"Caroline","family":"B\u00f6nisch","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9010-2080","authenticated-orcid":false,"given":"Christian","family":"Schmidt","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2166-846X","authenticated-orcid":false,"given":"Dorothea","family":"Keszty\u00fcs","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4759-5254","authenticated-orcid":false,"given":"Hans A","family":"Kestler","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0813-2393","authenticated-orcid":false,"given":"Tibor","family":"Keszty\u00fcs","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1010","published-online":{"date-parts":[[2025,6,30]]},"reference":[{"issue":"7023","key":"R1","doi-asserted-by":"publisher","first-page":"71","DOI":"10.1136\/bmj.312.7023.71","article-title":"Evidence based medicine: what it is and what it isn\u2019t","volume":"312","author":"Sackett","journal-title":"BMJ"},{"key":"R2","unstructured":"U.S. Food and Drug Administration. Real-world evidence. 2024. URL: https:\/\/www.fda.gov\/science-research\/science-and-research-special-topics\/real-world-evidence [Accessed 13-11-2024]"},{"issue":"1","key":"R3","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1197\/jamia.M2273","article-title":"Toward a national framework for the secondary use of health data: an American Medical Informatics Association White Paper","volume":"14","author":"Safran","journal-title":"J Am Med Inform Assoc"},{"key":"R4","doi-asserted-by":"publisher","DOI":"10.3233\/SHTI190248","article-title":"Detecting systemic data quality issues in electronic health records","volume":"264","author":"Ta","journal-title":"Stud Health Technol Inform"},{"issue":"S 01","key":"R5","doi-asserted-by":"publisher","first-page":"e46","DOI":"10.3414\/ME18-13-0001","article-title":"German medical informatics initiative: unlocking data for research and health care","volume":"57","author":"Gehring","journal-title":"Methods Inf Med"},{"issue":"1","key":"R6","doi-asserted-by":"publisher","DOI":"10.1186\/s12911-023-02195-3","article-title":"FAIRness through automation: development of an automated medical data integration infrastructure for FAIR health data in a maximum care university hospital","volume":"23","author":"Parciak","journal-title":"BMC Med Inform Decis Mak"},{"issue":"10","key":"R7","doi-asserted-by":"publisher","first-page":"1272","DOI":"10.1177\/1060028013501992","article-title":"Missed drug therapy alerts as a consequence of incomplete electronic patient records in Dutch community pharmacies","volume":"47","author":"Floor-Schreudering","journal-title":"Ann Pharmacother"},{"issue":"1","key":"R8","doi-asserted-by":"publisher","DOI":"10.2196\/25440","article-title":"Understanding the nature of metadata: systematic review","volume":"24","author":"Ulrich","journal-title":"J Med Internet Res"},{"issue":"3","key":"R9","doi-asserted-by":"publisher","DOI":"10.14569\/IJACSA.2024.0150358","article-title":"Enhancing data warehouses security","volume":"15","author":"Alkhubouli","journal-title":"IJACSA"},{"key":"R10","doi-asserted-by":"crossref","unstructured":"Nalbach O Linn C Derouet M Werth D . Business information systems. In: Abramowicz W Paschke A , editors. Presented at: Lecture Notes in Business Information Processing. Jun 16, 2018:Springer. 13-42; [doi: 10.1007\/978-3-319-93931-5_3]","DOI":"10.1007\/978-3-319-93931-5_3"},{"key":"R11","doi-asserted-by":"publisher","DOI":"10.3233\/SHTI230935","article-title":"FAIR+R: making clinical data reliable through qualitative metadata","volume":"310","author":"B\u00f6nisch","journal-title":"Stud Health Technol Inform"},{"issue":"7","key":"R12","doi-asserted-by":"publisher","first-page":"1879","DOI":"10.1007\/s10845-022-01963-8","article-title":"Machine learning and deep learning based predictive quality in manufacturing: a systematic review","volume":"33","author":"Tercan","journal-title":"J Intell Manuf"},{"key":"R13","doi-asserted-by":"publisher","DOI":"10.1016\/j.aei.2020.101101","article-title":"Predictive model-based quality inspection using machine learning and edge cloud computing","volume":"45","author":"Schmitt","journal-title":"Advanced Engineering Informatics"},{"issue":"1","key":"R14","doi-asserted-by":"publisher","first-page":"144","DOI":"10.1136\/amiajnl-2011-000681","article-title":"Methods and dimensions of electronic health record data quality assessment: enabling reuse for clinical research","volume":"20","author":"Weiskopf","journal-title":"J Am Med Inform Assoc"},{"issue":"6","key":"R15","doi-asserted-by":"publisher","first-page":"1072","DOI":"10.1093\/jamia\/ocx033","article-title":"A longitudinal analysis of data quality in a large pediatric data research network","volume":"24","author":"Khare","journal-title":"J Am Med Inform Assoc"},{"key":"R16","doi-asserted-by":"publisher","DOI":"10.1016\/j.ijmedinf.2016.03.006","article-title":"Data quality assessment framework to assess electronic medical record data for use in research","volume":"90","author":"Reimer","journal-title":"Int J Med Inform"},{"issue":"1","key":"R17","doi-asserted-by":"publisher","DOI":"10.1186\/s12911-021-01458-1","article-title":"A method for interoperable knowledge-based data quality assessment","volume":"21","author":"Tute","journal-title":"BMC Med Inform Decis Mak"},{"issue":"3","key":"R18","doi-asserted-by":"publisher","first-page":"199","DOI":"10.1214\/ss\/1009213726","article-title":"Statistical modeling: the two cultures (with comments and a rejoinder by the author)","volume":"16","author":"Breiman","journal-title":"Statist Sci"},{"issue":"11","key":"R19","doi-asserted-by":"publisher","first-page":"218","DOI":"10.21037\/atm.2016.03.37","article-title":"Introduction to machine learning: k-nearest neighbors","volume":"4","author":"Zhang","journal-title":"Ann Transl Med"},{"issue":"2\/3","key":"R20","doi-asserted-by":"publisher","first-page":"131","DOI":"10.1023\/A:1007465528199","article-title":"Bayesian network classifiers","volume":"29","author":"Friedman","journal-title":"Mach Learn"},{"issue":"1","key":"R21","doi-asserted-by":"publisher","first-page":"81","DOI":"10.1007\/BF00116251","article-title":"Induction of decision trees","volume":"1","author":"Quinlan","journal-title":"Mach Learn"},{"issue":"1","key":"R22","doi-asserted-by":"publisher","first-page":"5","DOI":"10.1023\/A:1010933404324","article-title":"Random forests","volume":"45","author":"Breiman","journal-title":"Mach Learn"},{"key":"R23","doi-asserted-by":"crossref","unstructured":"Chen T Guestrin C . XGBoost: a scalable tree boosting system. Presented at: KDD \u201916: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining San Francisco, CA, USA; Aug 13-17, 2016. [doi: 10.1145\/2939672.2939785]","DOI":"10.1145\/2939672.2939785"},{"key":"R24","doi-asserted-by":"crossref","unstructured":"Shmilovici A . Support vector machines. In: Maimon O Rokach L , editors. Data Mining and Knowledge Discovery Handbook. Springer; 2005. [doi: 10.1007\/0-387-25465-X_12]","DOI":"10.1007\/0-387-25465-X_12"},{"issue":"1","key":"R25","doi-asserted-by":"publisher","first-page":"67","DOI":"10.1109\/4235.585893","article-title":"No free lunch theorems for optimization","volume":"1","author":"Wolpert","journal-title":"IEEE Trans Evol Computat"},{"issue":"17","key":"R26","doi-asserted-by":"publisher","first-page":"1445","DOI":"10.1001\/jama.2024.2988","article-title":"Data checks before registering study protocols for health care database analyses","volume":"331","author":"Wang","journal-title":"JAMA"},{"issue":"1","key":"R27","doi-asserted-by":"publisher","DOI":"10.1038\/s41598-022-08351-5","article-title":"Automated quality assessment of large digitised histology cohorts by artificial intelligence","volume":"12","author":"Haghighat","journal-title":"Sci Rep"},{"key":"R28","doi-asserted-by":"publisher","DOI":"10.55630\/mem.2024.53.067-075","article-title":"Custom data quality mechanism in data warehouse facilitated by data integrity checks","volume":"53","author":"Georgiev","journal-title":"MEM"},{"key":"R29","doi-asserted-by":"publisher","DOI":"10.1051\/e3sconf\/202129701069","article-title":"A generic metadata management model for heterogeneous sources in a data warehouse","volume":"297","author":"Oukhouya","journal-title":"E3S Web Conf"},{"key":"R30","unstructured":"Goodfellow I J Mirza M Xiao D Courville A Bengio Y . An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv. Preprint posted online on 2013. [doi: 10.48550\/ARXIV.1312.6211]"},{"issue":"12","key":"R31","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1109\/TKDE.2018.2876857","article-title":"Learning under concept drift: a review","volume":"31","author":"Lu","journal-title":"IEEE Trans Knowl Data Eng"},{"issue":"2","key":"R32","doi-asserted-by":"publisher","DOI":"10.1177\/14604582241252763","article-title":"Evaluating health information systems-related errors using the human, organization, process, technology-fit (HOPT-fit) framework","volume":"30","author":"Yusof","journal-title":"Health Informatics J"},{"issue":"3","key":"R33","doi-asserted-by":"publisher","first-page":"103","DOI":"10.1177\/1833358318774357","article-title":"Data quality: \u201cgarbage in \u2013 garbage out\u201d","volume":"47","author":"Kilkenny","journal-title":"HIM J"}],"container-title":["JMIR Medical Informatics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/medinform.jmir.org\/2025\/1\/e60204","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,7,31]],"date-time":"2026-07-31T22:32:00Z","timestamp":1785537120000},"score":1,"resource":{"primary":{"URL":"https:\/\/medinform.jmir.org\/2025\/1\/e60204"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,6,30]]},"references-count":33,"URL":"https:\/\/doi.org\/10.2196\/60204","relation":{},"ISSN":["2291-9694"],"issn-type":[{"value":"2291-9694","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,6,30]]},"article-number":"v13i7023e60204"}}