{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,18]],"date-time":"2026-08-18T02:32:10Z","timestamp":1787020330736,"version":"build-2736575974"},"reference-count":37,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2021,7,1]],"date-time":"2021-07-01T00:00:00Z","timestamp":1625097600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2021,7,1]],"date-time":"2021-07-01T00:00:00Z","timestamp":1625097600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["BMC Med Inform Decis Mak"],"published-print":{"date-parts":[[2021,12]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:sec>\n                    <jats:title>Background<\/jats:title>\n                    <jats:p>Clinical risk prediction models (CRPMs) use patient characteristics to estimate the probability of having or developing a particular disease and\/or outcome. While CRPMs are gaining in popularity, they have yet to be widely adopted in clinical practice. The lack of explainability and interpretability has limited their utility. Explainability is the extent of which a model\u2019s prediction process can be described. Interpretability is the degree to which a user can understand the predictions made by a model.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Methods<\/jats:title>\n                    <jats:p>The study aimed to demonstrate utility of patient similarity analytics in developing an explainable and interpretable CRPM. Data was extracted from the electronic medical records of patients with type-2 diabetes mellitus, hypertension and dyslipidaemia in a Singapore public primary care clinic. We used modified K-nearest neighbour which incorporated expert input, to develop a patient similarity model on this real-world training dataset (n\u2009=\u20097,041) and validated it on a testing dataset (n\u2009=\u20093,018). The results were compared using logistic regression, random forest (RF) and support vector machine (SVM) models from the same dataset. The patient similarity model was then implemented in a prototype system to demonstrate the identification, explainability and interpretability of similar patients and the prediction process.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>The patient similarity model (AUROC\u2009=\u20090.718) was comparable to the logistic regression (AUROC\u2009=\u20090.695), RF (AUROC\u2009=\u20090.764) and SVM models (AUROC\u2009=\u20090.766). We packaged the patient similarity model in a prototype web application. A proof of concept demonstrated how the application provided both quantitative and qualitative information, in the form of patient narratives. This information was used to better inform and influence clinical decision-making, such as getting a patient to agree to start insulin therapy.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Conclusions<\/jats:title>\n                    <jats:p>Patient similarity analytics is a feasible approach to develop an explainable and interpretable CRPM. While the approach is generalizable, it can be used to develop locally relevant information, based on the database it searches. Ultimately, such an approach can generate a more informative CRPMs which can be deployed as part of clinical decision support tools to better facilitate shared decision-making in clinical practice.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.1186\/s12911-021-01566-y","type":"journal-article","created":{"date-parts":[[2021,7,1]],"date-time":"2021-07-01T09:03:13Z","timestamp":1625130193000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":21,"title":["Patient similarity analytics for explainable clinical risk prediction"],"prefix":"10.1186","volume":"21","author":[{"given":"Hao Sen Andrew","family":"Fang","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ngiap Chuan","family":"Tan","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Wei Ying","family":"Tan","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ronald Wihal","family":"Oei","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Mong Li","family":"Lee","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Wynne","family":"Hsu","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2021,7,1]]},"reference":[{"key":"1566_CR1","unstructured":"Wee L, van Kuijk SMJ, Dankers FJWM, Traverso A, Welch M, Dekker A. Reporting standards and critical appraisal of prediction models. In: Kubben P, Dumontier M, Dekker A, editors. Fundamentals of clinical data science [Internet]. Cham (CH): Springer; 2019 [cited 2020 Dec 7]. Available from: http:\/\/www.ncbi.nlm.nih.gov\/books\/NBK543529\/"},{"issue":"1","key":"1566_CR2","doi-asserted-by":"publisher","first-page":"129","DOI":"10.1111\/jth.12262","volume":"11","author":"JMT Hendriksen","year":"2013","unstructured":"Hendriksen JMT, Geersing GJ, Moons KGM, deGroot JAH. Diagnostic and prognostic prediction models. J Thromb Haemost JTH. 2013;11(1):129\u201341.","journal-title":"J Thromb Haemost JTH."},{"issue":"Suppl 4","key":"1566_CR3","doi-asserted-by":"publisher","first-page":"S574","DOI":"10.21037\/jtd.2019.01.25","volume":"11","author":"ME Shipe","year":"2019","unstructured":"Shipe ME, Deppen SA, Farjah F, Grogan EL. Developing prediction models for clinical use using logistic regression: an overview. J Thorac Dis. 2019;11(Suppl 4):S574\u201384.","journal-title":"J Thorac Dis"},{"key":"1566_CR4","doi-asserted-by":"publisher","first-page":"m958","DOI":"10.1136\/bmj.m958","volume":"369","author":"E Mahmoudi","year":"2020","unstructured":"Mahmoudi E, Kamdar N, Kim N, Gonzales G, Singh K, Waljee AK. Use of electronic medical records in development and validation of risk prediction models of hospital readmission: systematic review. BMJ. 2020;369:m958.","journal-title":"BMJ."},{"issue":"1","key":"1566_CR5","doi-asserted-by":"publisher","first-page":"76","DOI":"10.1038\/s41416-019-0476-8","volume":"121","author":"J Louro","year":"2019","unstructured":"Louro J, Posso M, Hilton Boon M, Rom\u00e1n M, Domingo L, Castells X, et al. A systematic review and quality assessment of individualised breast cancer risk prediction models. Br J Cancer. 2019;121(1):76\u201385.","journal-title":"Br J Cancer"},{"key":"1566_CR6","doi-asserted-by":"crossref","unstructured":"Kaiser I, Pfahlberg AB, Uter W, Heppt MV, Veier\u00f8d MB, Gefeller O. risk prediction models for melanoma: a systematic review on the heterogeneity in model development and validation. Int J Environ Res Public Health. 2020 Oct 28;17(21).","DOI":"10.3390\/ijerph17217919"},{"issue":"10","key":"1566_CR7","doi-asserted-by":"publisher","first-page":"e496","DOI":"10.1016\/S2589-7500(20)30226-0","volume":"2","author":"AM Leeuwenberg","year":"2020","unstructured":"Leeuwenberg AM, Schuit E. Prediction models for COVID-19 clinical decision making. Lancet Digit Health. 2020;2(10):e496\u20137.","journal-title":"Lancet Digit Health"},{"issue":"5","key":"1566_CR8","first-page":"752","volume":"32","author":"FW Dekker","year":"2017","unstructured":"Dekker FW, Ramspek CL, van Diepen M. Con: most clinical risk scores are useless. Nephrol Dial Transplant Off Publ Eur Dial Transpl Assoc - Eur Ren Assoc. 2017;32(5):752\u20135.","journal-title":"Nephrol Dial Transplant Off Publ Eur Dial Transpl Assoc - Eur Ren Assoc"},{"issue":"9","key":"1566_CR9","doi-asserted-by":"publisher","first-page":"e20645","DOI":"10.2196\/20645","volume":"22","author":"R Li","year":"2020","unstructured":"Li R, Yin C, Yang S, Qian B, Zhang P. Marrying medical domain knowledge with deep learning on electronic health records: a deep visual analytics approach. J Med Internet Res. 2020;22(9):e20645.","journal-title":"J Med Internet Res."},{"key":"1566_CR10","doi-asserted-by":"publisher","first-page":"g7594","DOI":"10.1136\/bmj.g7594","volume":"350","author":"GS Collins","year":"2015","unstructured":"Collins GS, Reitsma JB, Altman DG, Moons KGM. Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): the TRIPOD statement. BMJ. 2015;350:g7594.","journal-title":"BMJ."},{"issue":"1","key":"1566_CR11","doi-asserted-by":"publisher","first-page":"44","DOI":"10.1038\/s41591-018-0300-7","volume":"25","author":"EJ Topol","year":"2019","unstructured":"Topol EJ. High-performance medicine: the convergence of human and artificial intelligence. Nat Med. 2019;25(1):44\u201356.","journal-title":"Nat Med"},{"issue":"1","key":"1566_CR12","doi-asserted-by":"publisher","first-page":"3852","DOI":"10.1038\/s41467-020-17431-x","volume":"11","author":"SM Lauritsen","year":"2020","unstructured":"Lauritsen SM, Kristensen M, Olsen MV, Larsen MS, Lauritsen KM, J\u00f8rgensen MJ, et al. Explainable artificial intelligence model to predict acute critical illness from electronic health records. Nat Commun. 2020;11(1):3852.","journal-title":"Nat Commun."},{"issue":"4","key":"1566_CR13","doi-asserted-by":"publisher","first-page":"e0231166","DOI":"10.1371\/journal.pone.0231166","volume":"15","author":"E Zihni","year":"2020","unstructured":"Zihni E, Madai VI, Livne M, Galinovic I, Khalil AA, Fiebach JB, et al. Opening the black box of artificial intelligence for clinical decision support: A study predicting stroke outcome. PLOS ONE. 2020;15(4):e0231166.","journal-title":"PLOS ONE."},{"issue":"4","key":"1566_CR14","doi-asserted-by":"publisher","first-page":"e179","DOI":"10.1016\/S2589-7500(20)30018-2","volume":"2","author":"HC Thorsen-Meyer","year":"2020","unstructured":"Thorsen-Meyer HC, Nielsen AB, Nielsen AP, Kaas-Hansen BS, Toft P, Schierbeck J, et al. Dynamic and explainable machine learning prediction of mortality in patients in the intensive care unit: a retrospective study of high-frequency data in electronic patient records. Lancet Digit Health. 2020 Apr;2(4):e179\u201391.","journal-title":"Lancet Digit Health"},{"issue":"1","key":"1566_CR15","doi-asserted-by":"publisher","first-page":"198","DOI":"10.1093\/jamia\/ocw042","volume":"24","author":"BA Goldstein","year":"2017","unstructured":"Goldstein BA, Navar AM, Pencina MJ, Ioannidis JPA. Opportunities and challenges in developing risk prediction models with electronic health records data: a systematic review. J Am Med Inform Assoc JAMIA. 2017;24(1):198\u2013208.","journal-title":"J Am Med Inform Assoc JAMIA"},{"issue":"1","key":"1566_CR16","doi-asserted-by":"publisher","first-page":"1879","DOI":"10.1038\/s41598-019-38491-0","volume":"9","author":"B Shickel","year":"2019","unstructured":"Shickel B, Loftus TJ, Adhikari L, Ozrazgat-Baslanti T, Bihorac A, Rashidi P. DeepSOFA: a continuous acuity score for critically Ill patients using clinically interpretable deep learning. Sci Rep. 2019;9(1):1879.","journal-title":"Sci Rep."},{"key":"1566_CR17","doi-asserted-by":"publisher","first-page":"65333","DOI":"10.1109\/ACCESS.2018.2875677","volume":"6","author":"J Zhang","year":"2018","unstructured":"Zhang J, Kowsari K, Harrison JH, Lobo JM, Barnes LE. Patient2Vec: a personalized interpretable deep representation of the longitudinal electronic health record. IEEE Access. 2018;6:65333\u201346.","journal-title":"IEEE Access"},{"key":"1566_CR18","doi-asserted-by":"publisher","first-page":"18","DOI":"10.1038\/s41746-018-0029-1","volume":"1","author":"A Rajkomar","year":"2018","unstructured":"Rajkomar A, Oren E, Chen K, Dai AM, Hajaj N, Hardt M, et al. Scalable and accurate deep learning with electronic health records. NPJ Digit Med. 2018;1:18.","journal-title":"NPJ Digit Med"},{"issue":"1","key":"1566_CR19","doi-asserted-by":"publisher","first-page":"56","DOI":"10.1038\/s42256-019-0138-9","volume":"2","author":"SM Lundberg","year":"2020","unstructured":"Lundberg SM, Erion G, Chen H, DeGrave A, Prutkin JM, Nair B, et al. From local explanations to global understanding with explainable ai for trees. Nat Mach Intell. 2020;2(1):56\u201367.","journal-title":"Nat Mach Intell"},{"key":"1566_CR20","doi-asserted-by":"crossref","unstructured":"Bibal A, Lognoul M, Streel A, Fr\u00e9nay B. Legal requirements on explainability in machine learning. Artif Intell Law. 2020 Jul 30;","DOI":"10.1007\/s10506-020-09270-4"},{"issue":"2","key":"1566_CR21","doi-asserted-by":"publisher","first-page":"167","DOI":"10.1093\/jamiaopen\/ooz046","volume":"3","author":"J Watson","year":"2020","unstructured":"Watson J, Hutyra CA, Clancy SM, Chandiramani A, Bedoya A, Ilangovan K, et al. Overcoming barriers to the adoption and implementation of predictive modeling and machine learning in clinical care: what can we learn from US academic medical centers? JAMIA Open. 2020;3(2):167\u201372.","journal-title":"JAMIA Open"},{"issue":"5","key":"1566_CR22","doi-asserted-by":"crossref","first-page":"e1379","DOI":"10.1002\/widm.1379","volume":"10","author":"G Stiglic","year":"2020","unstructured":"Stiglic G, Kocbek P, Fijacko N, Zitnik M, Verbert K, Cilar L. Interpretability of machine learning-based prediction models in healthcare. WIREs Data Min Knowl Discov. 2020;10(5):e1379.","journal-title":"WIREs Data Min Knowl Discov."},{"key":"1566_CR23","doi-asserted-by":"publisher","first-page":"561","DOI":"10.3389\/fphys.2016.00561","volume":"7","author":"S-A Brown","year":"2016","unstructured":"Brown S-A. Patient similarity: emerging concepts in systems and precision medicine. Front Physiol. 2016;7:561.","journal-title":"Front Physiol"},{"key":"1566_CR24","doi-asserted-by":"publisher","first-page":"87","DOI":"10.1016\/j.jbi.2018.06.001","volume":"83","author":"E Parimbelli","year":"2018","unstructured":"Parimbelli E, Marini S, Sacchi L, Bellazzi R. Patient similarity for precision medicine: a systematic review. J Biomed Inf. 2018;83:87\u201396.","journal-title":"J Biomed Inf"},{"key":"1566_CR25","doi-asserted-by":"crossref","unstructured":"Hassan S, Syed Z. From netflix to heart attacks: Collaborative filtering in medical datasets. IHI\u201910 - Proceedings of the 1st ACM International Health Informatics Symposium. 2010. 128 p.","DOI":"10.1145\/1882992.1883012"},{"issue":"1","key":"1566_CR26","doi-asserted-by":"publisher","first-page":"98","DOI":"10.1186\/s12938-019-0718-2","volume":"18","author":"N Wang","year":"2019","unstructured":"Wang N, Huang Y, Liu H, Fei X, Wei L, Zhao X, et al. Measurement and application of patient similarity in personalized predictive modeling based on electronic medical records. Biomed Eng OnLine. 2019;18(1):98.","journal-title":"Biomed Eng OnLine"},{"issue":"9","key":"1566_CR27","doi-asserted-by":"publisher","first-page":"604","DOI":"10.7326\/0003-4819-150-9-200905050-00006","volume":"150","author":"AS Levey","year":"2009","unstructured":"Levey AS, Stevens LA, Schmid CH, Zhang YL, Castro AF, Feldman HI, et al. A new equation to estimate glomerular filtration rate. Ann Int Med. 2009;150(9):604\u201312.","journal-title":"Ann Int Med"},{"issue":"4","key":"1566_CR28","doi-asserted-by":"publisher","first-page":"026591","DOI":"10.1136\/bmjopen-2018-026591","volume":"9","author":"J Dyson","year":"2019","unstructured":"Dyson J, Marsh C, Jackson N, Richardson D, Faisal M, Scally AJ, et al. Understanding and applying practitioner and patient views on the implementation of a novel automated Computer-Aided Risk Score (CARS) predicting the risk of death following emergency medical admission to hospital: qualitative study. BMJ Open. 2019;9(4):026591.","journal-title":"BMJ Open."},{"issue":"3","key":"1566_CR29","doi-asserted-by":"publisher","first-page":"455","DOI":"10.1016\/j.pec.2012.06.012","volume":"88","author":"GM Fix","year":"2012","unstructured":"Fix GM, Houston TK, Barker AM, Wexler L, Cook N, Volkman JE, et al. A novel process for integrating patient stories into patient education interventions: incorporating lessons from theater arts. Patient Educ Couns. 2012;88(3):455\u20139.","journal-title":"Patient Educ Couns"},{"issue":"9","key":"1566_CR30","doi-asserted-by":"publisher","first-page":"1482","DOI":"10.1016\/j.pec.2016.06.024","volume":"99","author":"BG Bokhour","year":"2016","unstructured":"Bokhour BG, Fix GM, Gordon HS, Long JA, DeLaughter K, Orner MB, et al. Can stories influence African-American patients\u2019 intentions to change hypertension management behaviors? A randomized control trial. Patient Educ Couns. 2016;99(9):1482\u20138.","journal-title":"Patient Educ Couns"},{"issue":"265","key":"1566_CR31","first-page":"175","volume":"9","author":"B Lesselroth","year":"2019","unstructured":"Lesselroth B, Monkman H. Narratives and stories: novel approaches to improving patient-facing information resources and patient engagement. Stud Health Technol Inform. 2019;9(265):175\u201380.","journal-title":"Stud Health Technol Inform"},{"key":"1566_CR32","first-page":"160","volume":"1","author":"D Hosmer","year":"2000","unstructured":"Hosmer D, Lemeshow S. Area under the ROC curve. Appl Logist Regres. 2000;1:160\u20134.","journal-title":"Appl Logist Regres"},{"issue":"2","key":"1566_CR33","doi-asserted-by":"publisher","first-page":"295","DOI":"10.1177\/1932296817706375","volume":"12","author":"A Dagliati","year":"2018","unstructured":"Dagliati A, Marini S, Sacchi L, Cogni G, Teliti M, Tibollo V, et al. Machine learning methods to predict diabetes complications. J Diabetes Sci Technol. 2018;12(2):295\u2013302.","journal-title":"J Diabetes Sci Technol"},{"key":"1566_CR34","first-page":"6","volume":"10","author":"Y Bashiri","year":"2018","unstructured":"Bashiri Y, Arani M, Bagheri N. Investigating factors associated with diabetes complications among Type 2 diabetic patients. J Res Med Dent Sci. 2018 May;10:6.","journal-title":"J Res Med Dent Sci"},{"issue":"1","key":"1566_CR35","doi-asserted-by":"publisher","first-page":"25","DOI":"10.1089\/big.2015.0029","volume":"4","author":"W Oh","year":"2016","unstructured":"Oh W, Kim E, Castro MR, Caraballo PJ, Kumar V, Steinbach MS, et al. Type 2 diabetes mellitus trajectories and associated risks. Big Data. 2016 Mar 1;4(1):25\u201330.","journal-title":"Big Data"},{"issue":"21","key":"1566_CR36","doi-asserted-by":"publisher","first-page":"013479","DOI":"10.1161\/JAHA.119.013479","volume":"8","author":"A Dayimu","year":"2019","unstructured":"Dayimu A, Wang C, Li J, Fan B, Ji X, Zhang T, et al. Trajectories of lipids profile and incident cardiovascular disease risk: a longitudinal cohort study. J Am Heart Assoc. 2019;8(21):013479.","journal-title":"J Am Heart Assoc."},{"key":"1566_CR37","doi-asserted-by":"publisher","first-page":"119","DOI":"10.1016\/j.jbi.2016.03.009","volume":"61","author":"DM Vock","year":"2016","unstructured":"Vock DM, Wolfson J, Bandyopadhyay S, Adomavicius G, Johnson PE, Vazquez-Benitez G, et al. Adapting machine learning techniques to censored time-to-event health record data: A general-purpose approach using inverse probability of censoring weighting. J Biomed Inform. 2016 Jun;61:119\u201331.","journal-title":"J Biomed Inform"}],"container-title":["BMC Medical Informatics and Decision Making"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s12911-021-01566-y.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1186\/s12911-021-01566-y\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s12911-021-01566-y.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,11,5]],"date-time":"2023-11-05T09:30:22Z","timestamp":1699176622000},"score":1,"resource":{"primary":{"URL":"https:\/\/bmcmedinformdecismak.biomedcentral.com\/articles\/10.1186\/s12911-021-01566-y"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,7,1]]},"references-count":37,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2021,12]]}},"alternative-id":["1566"],"URL":"https:\/\/doi.org\/10.1186\/s12911-021-01566-y","relation":{},"ISSN":["1472-6947"],"issn-type":[{"value":"1472-6947","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,7,1]]},"assertion":[{"value":"14 January 2021","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"22 June 2021","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"1 July 2021","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"Ethics board approval was obtained from the SingHealth Centralized Institutional Review Board (Reference Number: 2019\/2604) prior to conduct of the study. The requirement of written consent was also waived by the SingHealth Centralized Institutional Review Board as it was deemed impracticable while privacy risks were mitigated through the use of de-identified data. All methods were carried out in accordance with relevant guidelines and regulations.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethics approval and consent to participate"}},{"value":"Not applicable.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Consent for publication"}},{"value":"The authors declare no competing interests.","order":4,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"207"}}