{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,4]],"date-time":"2026-04-04T04:14:39Z","timestamp":1775276079632,"version":"3.50.1"},"reference-count":39,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2024,6,24]],"date-time":"2024-06-24T00:00:00Z","timestamp":1719187200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2024,6,24]],"date-time":"2024-06-24T00:00:00Z","timestamp":1719187200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["BMC Med Inform Decis Mak"],"abstract":"<jats:title>Abstract<\/jats:title><jats:sec>\n                <jats:title>Objective<\/jats:title>\n                <jats:p>This study aimed to develop and validate a quantitative index system for evaluating the data quality of Electronic Medical Records (EMR) in disease risk prediction using Machine Learning (ML).<\/jats:p>\n              <\/jats:sec><jats:sec>\n                <jats:title>Materials and methods<\/jats:title>\n                <jats:p>The index system was developed in four steps: (1) a preliminary index system was outlined based on literature review; (2) we utilized the Delphi method to structure the indicators at all levels; (3) the weights of these indicators were determined using the Analytic Hierarchy Process (AHP) method; and (4) the developed index system was empirically validated using real-world EMR data in a ML-based disease risk prediction task.<\/jats:p>\n              <\/jats:sec><jats:sec>\n                <jats:title>Results<\/jats:title>\n                <jats:p>The synthesis of review findings and the expert consultations led to the formulation of a three-level index system with four first-level, 11\u00a0second-level, and 33 third-level indicators. The weights of these indicators were obtained through the AHP method. Results from the empirical analysis illustrated a positive relationship between the scores assigned by the proposed index system and the predictive performances of the datasets.<\/jats:p>\n              <\/jats:sec><jats:sec>\n                <jats:title>Discussion<\/jats:title>\n                <jats:p>The proposed index system for evaluating EMR data quality is grounded in extensive literature analysis and expert consultation. Moreover, the system\u2019s high reliability and suitability has been affirmed through empirical validation.<\/jats:p>\n              <\/jats:sec><jats:sec>\n                <jats:title>Conclusion<\/jats:title>\n                <jats:p>The novel index system offers a robust framework for assessing the quality and suitability of EMR data in ML-based disease risk predictions. It can serve as a guide in building EMR databases, improving EMR data quality control, and generating reliable real-world evidence.<\/jats:p>\n              <\/jats:sec>","DOI":"10.1186\/s12911-024-02533-z","type":"journal-article","created":{"date-parts":[[2024,6,24]],"date-time":"2024-06-24T10:02:05Z","timestamp":1719223325000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":7,"title":["Development of a quantitative index system for evaluating the quality of electronic medical records in disease risk intelligent prediction"],"prefix":"10.1186","volume":"24","author":[{"given":"Jiayin","family":"Zhou","sequence":"first","affiliation":[]},{"given":"Jie","family":"Hao","sequence":"additional","affiliation":[]},{"given":"Mingkun","family":"Tang","sequence":"additional","affiliation":[]},{"given":"Haixia","family":"Sun","sequence":"additional","affiliation":[]},{"given":"Jiayang","family":"Wang","sequence":"additional","affiliation":[]},{"given":"Jiao","family":"Li","sequence":"additional","affiliation":[]},{"given":"Qing","family":"Qian","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2024,6,24]]},"reference":[{"issue":"1","key":"2533_CR1","doi-asserted-by":"publisher","first-page":"10","DOI":"10.1002\/cpt.1295","volume":"105","author":"SA Waldman","year":"2019","unstructured":"Waldman SA, Terzic A. Healthcare evolves from reactive to proactive. Clin Pharmacol Ther. 2019;105(1):10.","journal-title":"Clin Pharmacol Ther"},{"key":"2533_CR2","doi-asserted-by":"publisher","first-page":"4417","DOI":"10.1007\/s00521-019-04095-y","volume":"32","author":"MI Razzak","year":"2020","unstructured":"Razzak MI, Imran M, Xu G. Big data analytics for preventive medicine. Neural Comput Appl. 2020;32:4417\u201351.","journal-title":"Neural Comput Appl"},{"issue":"2","key":"2533_CR3","doi-asserted-by":"publisher","first-page":"94","DOI":"10.7861\/futurehosp.6-2-94","volume":"6","author":"T Davenport","year":"2019","unstructured":"Davenport T, Kalakota R. The potential for artificial intelligence in healthcare. Future Healthc J. 2019;6(2):94\u20138.","journal-title":"Future Healthc J"},{"issue":"463","key":"2533_CR4","doi-asserted-by":"publisher","first-page":"eaau4778","DOI":"10.1126\/scitranslmed.aau4778","volume":"10","author":"VJ Dzau","year":"2018","unstructured":"Dzau VJ, Balatbat CA. Health and societal implications of medical and technological advances. Sci Transl Med. 2018;10(463):eaau4778.","journal-title":"Sci Transl Med."},{"key":"2533_CR5","volume-title":"The Computer-Based Patient Record: An Essential Technology for Health Care","author":"Institute of Medicine","year":"1997","unstructured":"Institute of Medicine. The Computer-Based Patient Record: An Essential Technology for Health Care. Washington DC: National Academy Press; 1997."},{"issue":"2","key":"2533_CR6","doi-asserted-by":"publisher","first-page":"57","DOI":"10.1200\/jop.2005.1.2.57","volume":"1","author":"EP Ambinder","year":"2005","unstructured":"Ambinder EP. Electronic health records. Journal of oncology practice. 2005;1(2):57.","journal-title":"Journal of oncology practice"},{"issue":"7","key":"2533_CR7","first-page":"500","volume":"38","author":"M Motwani","year":"2017","unstructured":"Motwani M, Dey D, Berman DS, et al. Machine learning for prediction of all-cause mortality in patients with suspected coronary artery disease: a 5-year multi centre prospective registry analysis. Eur Heart J. 2017;38(7):500\u20137.","journal-title":"Eur Heart J"},{"issue":"1","key":"2533_CR8","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0169772","volume":"12","author":"J Allyn","year":"2017","unstructured":"Allyn J, Allou N, Augustin P, et al. A comparison of a machine learning model with EuroSCORE II in predicting mortality after elective cardiac surgery: a decision curve analysis. PLoS ONE. 2017;12(1): e0169772.","journal-title":"PLoS ONE"},{"key":"2533_CR9","first-page":"320","volume":"1998","author":"A Geissbuhler","year":"1998","unstructured":"Geissbuhler A, Miller RA. Clinical application of the UMLS in a computerized order entry and decision-support system. Proc AMIA Symp. 1998;1998:320\u20134.","journal-title":"Proc AMIA Symp"},{"issue":"2","key":"2533_CR10","doi-asserted-by":"publisher","first-page":"84","DOI":"10.1089\/omi.2006.10.84","volume":"10","author":"D Field","year":"2006","unstructured":"Field D, Sansone SA. A special issue on data standards. OMICS. 2006;10(2):84\u201393.","journal-title":"OMICS"},{"issue":"1","key":"2533_CR11","first-page":"71","volume":"20","author":"CN Mead","year":"2006","unstructured":"Mead CN. Data interchange standards in healthcare IT\u2013computable semantic interoperability: now possible but still difficult, do we really need a better mousetrap? J Healthc Inf Manag. 2006;20(1):71\u20138.","journal-title":"J Healthc Inf Manag"},{"key":"2533_CR12","doi-asserted-by":"publisher","first-page":"40","DOI":"10.1016\/j.ijmedinf.2016.03.006","volume":"90","author":"AP Reimer","year":"2016","unstructured":"Reimer AP, Milinovich A, Madigan EA. Data quality assessment framework to assess electronic medical record data for use in research. Int J Med Inform. 2016;90:40\u20137.","journal-title":"Int J Med Inform"},{"issue":"1","key":"2533_CR13","doi-asserted-by":"publisher","first-page":"69","DOI":"10.4338\/ACI-2015-08-RA-0107","volume":"7","author":"SG Johnson","year":"2016","unstructured":"Johnson SG, Speedie S, Simon G, et al. Application of an ontology for characterizing data quality for a secondary use of EHR data. Appl Clin Inform. 2016;7(1):69\u201388.","journal-title":"Appl Clin Inform"},{"issue":"1","key":"2533_CR14","doi-asserted-by":"publisher","first-page":"23","DOI":"10.1007\/s10916-022-01892-2","volume":"47","author":"O Ozonze","year":"2023","unstructured":"Ozonze O, Scott PJ, Hopgood AA. Automating electronic health record data quality assessment. J Med Syst. 2023;47(1):23.","journal-title":"J Med Syst"},{"key":"2533_CR15","unstructured":"Strasser A. Delphi method variants in IS research: a taxonomy proposal. In: PAC15 2016 Proceedings. 2016. https:\/\/aisel.aisnet.org\/pacis2016\/224. Accessed 5 July 2023."},{"key":"2533_CR16","volume-title":"The Practice of Social Research","author":"ER Babbie","year":"2014","unstructured":"Babbie ER. The Practice of Social Research. Mason, OH: CENGAGE Learning Custom Publishing; 2014."},{"key":"2533_CR17","volume-title":"Social Research Methods","author":"A Bryman","year":"2015","unstructured":"Bryman A. Social Research Methods. London, England: Oxford University Press; 2015."},{"key":"2533_CR18","doi-asserted-by":"publisher","first-page":"1004910","DOI":"10.3389\/fpubh.2022.1004910","volume":"10","author":"Y Ruan","year":"2022","unstructured":"Ruan Y, Song S, Yin Z, et al. Comprehensive evaluation of military training-induced fatigue among soldiers in China: A Delphi consensus study. Front Public Health. 2022;10:1004910.","journal-title":"Front Public Health"},{"issue":"3","key":"2533_CR19","doi-asserted-by":"publisher","first-page":"161","DOI":"10.1016\/0038-0121(89)90013-X","volume":"23","author":"JP Shim","year":"1989","unstructured":"Shim JP. Bibliographical research on the analytic hierarchy process (AHP). Socio-Econ Plann Sci. 1989;23(3):161\u20137.","journal-title":"Socio-Econ Plann Sci"},{"issue":"1","key":"2533_CR20","doi-asserted-by":"publisher","first-page":"211","DOI":"10.1016\/j.ejor.2007.01.004","volume":"186","author":"W Ho","year":"2008","unstructured":"Ho W. Integrated analytic hierarchy process and its applications\u2013A literature review. Eur J Oper Res. 2008;186(1):211\u201328.","journal-title":"Eur J Oper Res"},{"issue":"3","key":"2533_CR21","doi-asserted-by":"publisher","first-page":"575","DOI":"10.1111\/j.1540-5915.1989.tb01568.x","volume":"20","author":"EF Lane","year":"1989","unstructured":"Lane EF, Verdini WA. A consistency test for AHP decision makers. Decis Sci. 1989;20(3):575\u201390.","journal-title":"Decis Sci"},{"key":"2533_CR22","doi-asserted-by":"publisher","unstructured":"Johnson AE, Pollard TJ, Mark RG. \u201cMIMIC-III clinical database (version 1.4),\u201d PhysioNet. 2016; https:\/\/doi.org\/10.13026\/C2XW26.","DOI":"10.13026\/C2XW26"},{"key":"2533_CR23","doi-asserted-by":"publisher","first-page":"56","DOI":"10.1016\/j.jclinepi.2020.03.002","volume":"122","author":"S Nusinovici","year":"2020","unstructured":"Nusinovici S, Tham YC, Yan MYC, et al. Logistic regression was as good as machine learning for predicting major chronic diseases. J Clin Epidemiol. 2020;122:56\u201369.","journal-title":"J Clin Epidemiol"},{"key":"2533_CR24","doi-asserted-by":"publisher","first-page":"183","DOI":"10.1016\/j.neuroimage.2014.03.067","volume":"96","author":"T Watanabe","year":"2014","unstructured":"Watanabe T, Kessler D, Scott C, et al. Disease prediction based on functional connectomes using a scalable and spatially-informed support vector machine. Neuroimage. 2014;96:183\u2013202.","journal-title":"Neuroimage"},{"issue":"1","key":"2533_CR25","doi-asserted-by":"publisher","first-page":"5245","DOI":"10.1038\/s41598-020-62133-5","volume":"10","author":"L Yang","year":"2020","unstructured":"Yang L, Wu H, Jin X, et al. Study of cardiovascular disease prediction model based on random forest in eastern China. Sci Rep. 2020;10(1):5245.","journal-title":"Sci Rep"},{"key":"2533_CR26","first-page":"2825","volume":"12","author":"F Pedregosa","year":"2011","unstructured":"Pedregosa F, Varoquaux G, Gramfort A, et al. Scikit-learn: Machine learning in Python. J Machine Learn Res. 2011;12:2825\u201330.","journal-title":"J Machine Learn Res."},{"key":"2533_CR27","doi-asserted-by":"crossref","unstructured":"Cohen I, Huang Y, Chen J, et al. Pearson correlation coefficient. In: Noise reduction in speech processing. Heidelberg: Springer; 2009:1\u20134.","DOI":"10.1007\/978-3-642-00296-0_5"},{"issue":"2","key":"2533_CR28","doi-asserted-by":"publisher","first-page":"87","DOI":"10.2478\/v10117-011-0021-1","volume":"30","author":"J Hauke","year":"2011","unstructured":"Hauke J, Kossowski T. Comparison of values of Pearson\u2019s and Spearman\u2019s correlation coefficients on the same sets of data. Quaestion Geograph. 2011;30(2):87\u201393.","journal-title":"Quaestion Geograph"},{"issue":"1","key":"2533_CR29","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1038\/sdata.2016.35","volume":"3","author":"AEW Johnson","year":"2016","unstructured":"Johnson AEW, Pollard TJ, Shen L, et al. MIMIC-III, a freely accessible critical care database. Scientific data. 2016;3(1):1\u20139.","journal-title":"Scientific data"},{"issue":"1","key":"2533_CR30","doi-asserted-by":"publisher","first-page":"4","DOI":"10.4161\/viru.27372","volume":"5","author":"FB Mayr","year":"2014","unstructured":"Mayr FB, Yende S, Angus DC. Epidemiology of severe sepsis. Virulence. 2014;5(1):4\u201311.","journal-title":"Virulence"},{"issue":"28","key":"2533_CR31","doi-asserted-by":"publisher","DOI":"10.1097\/MD.0000000000011421","volume":"97","author":"P Lan","year":"2018","unstructured":"Lan P, Wang SJ, Shi QC, et al. Comparison of the predictive value of scoring systems on the prognosis of cirrhotic patients with suspected infection. Medicine. 2018;97(28): e11230.","journal-title":"Medicine"},{"issue":"1","key":"2533_CR32","doi-asserted-by":"publisher","first-page":"11072","DOI":"10.1038\/s41598-018-29353-2","volume":"8","author":"P Lan","year":"2018","unstructured":"Lan P, Pan K, Wang S, et al. High serum iron level is associated with increased mortality in patients with sepsis. Sci Rep. 2018;8(1):11072.","journal-title":"Sci Rep"},{"issue":"1","key":"2533_CR33","doi-asserted-by":"publisher","first-page":"145","DOI":"10.1093\/bioinformatics\/btw570","volume":"33","author":"T Saito","year":"2017","unstructured":"Saito T, Rehmsmeier M. Precrec: fast and accurate precision-recall and ROC curve calculations in R. Bioinformatics. 2017;33(1):145\u20137.","journal-title":"Bioinformatics"},{"issue":"Suppl 1","key":"2533_CR34","doi-asserted-by":"publisher","first-page":"79","DOI":"10.1080\/12460125.2020.1776927","volume":"29","author":"A Ferencek","year":"2020","unstructured":"Ferencek A, Kljaji\u0107 BM. Data quality assessment in product failure prediction models. J Decis Syst. 2020;29(Suppl 1):79\u201386.","journal-title":"J Decis Syst"},{"key":"2533_CR35","first-page":"1937","volume":"2015","author":"SG Johnson","year":"2015","unstructured":"Johnson SG, Speedie S, Simon G, et al. A data quality ontology for the secondary use of EHR data. AMIA Annu Symp Proc. 2015;2015:1937\u201346.","journal-title":"AMIA Annu Symp Proc"},{"issue":"10","key":"2533_CR36","first-page":"9","volume":"41","author":"Q Tian","year":"2020","unstructured":"Tian Q, Chen Y, Han Z, et al. Research on evaluation indexes of clinical data quality. J Med Inform. 2020;41(10):9\u201317.","journal-title":"J Med Inform"},{"issue":"1","key":"2533_CR37","first-page":"14","volume":"5","author":"NG Weiskopf","year":"2017","unstructured":"Weiskopf NG, Bakken S, Hripcsak G, et al. A data quality assessment guideline for electronic health record data reuse. EGEMS (Wash DC). 2017;5(1):14.","journal-title":"EGEMS (Wash DC)"},{"issue":"1","key":"2533_CR38","first-page":"1244","volume":"4","author":"MG Kahn","year":"2016","unstructured":"Kahn MG, Callahan TJ, Barnard J, et al. A harmonized data quality assessment terminology and framework for the secondary use of electronic health record data. EGEMS (Wash DC). 2016;4(1):1244.","journal-title":"EGEMS (Wash DC)"},{"key":"2533_CR39","doi-asserted-by":"publisher","first-page":"2","DOI":"10.5334\/dsj-2015-002","volume":"14","author":"L Cai","year":"2015","unstructured":"Cai L, Zhu Y. The challenges of data quality and data quality assessment in the big data era. Data Sci J. 2015;14:2\u20132.","journal-title":"Data Sci J"}],"container-title":["BMC Medical Informatics and Decision Making"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s12911-024-02533-z.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1186\/s12911-024-02533-z\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s12911-024-02533-z.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,6,24]],"date-time":"2024-06-24T10:04:02Z","timestamp":1719223442000},"score":1,"resource":{"primary":{"URL":"https:\/\/bmcmedinformdecismak.biomedcentral.com\/articles\/10.1186\/s12911-024-02533-z"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,6,24]]},"references-count":39,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2024,12]]}},"alternative-id":["2533"],"URL":"https:\/\/doi.org\/10.1186\/s12911-024-02533-z","relation":{},"ISSN":["1472-6947"],"issn-type":[{"value":"1472-6947","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,6,24]]},"assertion":[{"value":"5 July 2023","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"13 May 2024","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"24 June 2024","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"Ethics approval was not deemed necessary for this study by the Ethics Committee at the Institute of Medical Information & Library, Chinese Academy of Medical Sciences, in accordance with national guidelines and local legislation. Written informed consent was obtained from all participants. All methods were performed in accordance with the relevant guidelines and regulations. This study was conducted in compliance with the Declaration of the Helsinki.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethics approval and consent to participate"}},{"value":"Not applicable.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Consent for publication"}},{"value":"The authors declare no competing interests.","order":4,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"178"}}