{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,28]],"date-time":"2026-08-28T06:26:46Z","timestamp":1787898406646,"version":"build-2784847793"},"reference-count":38,"publisher":"MDPI AG","issue":"6","license":[{"start":{"date-parts":[[2021,5,29]],"date-time":"2021-05-29T00:00:00Z","timestamp":1622246400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Diagnostics"],"abstract":"<jats:p>Laboratory tests are performed to make effective clinical decisions. However, inappropriate laboratory test ordering hampers patient care and increases financial burden for healthcare. An automated laboratory test recommendation system can provide rapid and appropriate test selection, potentially improving the workflow to help physicians spend more time treating patients. The main objective of this study was to develop a deep learning-based automated system to recommend appropriate laboratory tests. A retrospective data collection was performed at the National Health Insurance database between 1 January 2013, and 31 December 2013. We included all prescriptions that had at least one laboratory test. A total of 1,463,837 prescriptions from 530,050 unique patients was included in our study. Of these patients, 296,541 were women (55.95%), the range of age was between 1 and 107 years. The deep learning (DL) model achieved a higher area under the receiver operating characteristics curve (AUROC micro = 0.98, and AUROC macro = 0.94). The findings of this study show that the DL model can accurately and efficiently identify laboratory tests. This model can be integrated into existing workflows to reduce under- and over-utilization problems.<\/jats:p>","DOI":"10.3390\/diagnostics11060990","type":"journal-article","created":{"date-parts":[[2021,5,31]],"date-time":"2021-05-31T00:22:15Z","timestamp":1622420535000},"page":"990","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":29,"title":["Deep into Laboratory: An Artificial Intelligence Approach to Recommend Laboratory Tests"],"prefix":"10.3390","volume":"11","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-6026-2748","authenticated-orcid":false,"given":"Md. Mohaimenul","family":"Islam","sequence":"first","affiliation":[{"name":"Graduate Institute of Biomedical Informatics, College of Medical Science and Technology, Taipei Medical University, Taipei 110, Taiwan"},{"name":"International Center for Health Information Technology (ICHIT), Taipei Medical University, Taipei 110, Taiwan"},{"name":"Research Center of Big Data and Meta-Analysis, Wan Fang Hospital, Taipei Medical University, Taipei 116, Taiwan"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Tahmina Nasrin","family":"Poly","sequence":"additional","affiliation":[{"name":"Graduate Institute of Biomedical Informatics, College of Medical Science and Technology, Taipei Medical University, Taipei 110, Taiwan"},{"name":"International Center for Health Information Technology (ICHIT), Taipei Medical University, Taipei 110, Taiwan"},{"name":"Research Center of Big Data and Meta-Analysis, Wan Fang Hospital, Taipei Medical University, Taipei 116, Taiwan"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9198-0697","authenticated-orcid":false,"given":"Hsuan-Chia","family":"Yang","sequence":"additional","affiliation":[{"name":"Graduate Institute of Biomedical Informatics, College of Medical Science and Technology, Taipei Medical University, Taipei 110, Taiwan"},{"name":"International Center for Health Information Technology (ICHIT), Taipei Medical University, Taipei 110, Taiwan"},{"name":"Research Center of Big Data and Meta-Analysis, Wan Fang Hospital, Taipei Medical University, Taipei 116, Taiwan"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6497-4232","authenticated-orcid":false,"given":"Yu-Chuan (Jack)","family":"Li","sequence":"additional","affiliation":[{"name":"Graduate Institute of Biomedical Informatics, College of Medical Science and Technology, Taipei Medical University, Taipei 110, Taiwan"},{"name":"International Center for Health Information Technology (ICHIT), Taipei Medical University, Taipei 110, Taiwan"},{"name":"Research Center of Big Data and Meta-Analysis, Wan Fang Hospital, Taipei Medical University, Taipei 116, Taiwan"},{"name":"Department of Dermatology, Wan Fang Hospital, Taipei 116, Taiwan"},{"name":"TMU Research Center of Cancer Translational Medicine, Taipei Medical University, Taipei 110, Taiwan"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2021,5,29]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"e80","DOI":"10.1309\/LMD0YIFPTOWZONAD","article-title":"Quality-improvement measures as effective ways of preventing laboratory errors","volume":"45","author":"Agarwal","year":"2014","journal-title":"Lab. Med."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"194","DOI":"10.5858\/arpa.2013-0693-OA","article-title":"A cost-effective interdisciplinary approach to microbiologic send-out test use","volume":"139","author":"Aesif","year":"2015","journal-title":"Arch. Pathol. Lab. Med."},{"key":"ref_3","first-page":"377","article-title":"Effect of the characteristics of family physicians on their utilisation of laboratory tests","volume":"57","author":"Vinker","year":"2007","journal-title":"Br. J. Gen. Pract."},{"key":"ref_4","first-page":"k4666","article-title":"Temporal trends in use of tests in UK primary care, 2000-15: Retrospective analysis of 250 million tests","volume":"363","author":"Stevens","year":"2018","journal-title":"BMJ"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"268","DOI":"10.3122\/jabfm.2014.02.130104","article-title":"Primary care physicians\u2019 challenges in ordering clinical laboratory tests and interpreting results","volume":"27","author":"Hickner","year":"2014","journal-title":"J. Am. Board Fam. Med."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Zhi, M., Ding, E.L., Theisen-Toupal, J., Whelan, J., and Arnaout, R. (2013). The landscape of inappropriate laboratory testing: A 15-year meta-analysis. PLoS ONE, 8.","DOI":"10.1371\/journal.pone.0078962"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"555","DOI":"10.1016\/j.clinbiochem.2017.03.002","article-title":"The role of laboratory in ensuring appropriate test requests","volume":"50","author":"Ferraro","year":"2017","journal-title":"Clin. Biochem."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Schumacher, L.D., J\u00e4ger, L., Meier, R., Rachamin, Y., Senn, O., Rosemann, T., and Markun, S. (2020). Trends and between-Physician Variation in Laboratory Testing: A Retrospective Longitudinal Study in General Practice. J. Clin. Med., 9.","DOI":"10.3390\/jcm9061787"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"1341","DOI":"10.2147\/IJGM.S275406","article-title":"Trends in Micronutrient Laboratory Testing in Switzerland: A 7-Year Retrospective Analysis of Healthcare Claims Data","volume":"13","author":"Huber","year":"2020","journal-title":"Int. J. Gen. Med."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"82","DOI":"10.21037\/atm.2017.02.04","article-title":"Inappropriateness in laboratory medicine: An elephant in the room?","volume":"5","author":"Lippi","year":"2017","journal-title":"Ann. Transl. Med."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"e1913325","DOI":"10.1001\/jamanetworkopen.2019.13325","article-title":"Cascades of care after incidental findings in a US national survey of physicians","volume":"2","author":"Ganguli","year":"2019","journal-title":"JAMA Netw. Open"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"48","DOI":"10.1016\/j.amsu.2020.02.002","article-title":"Inappropriate laboratory testing in internal medicine inpatients: Prevalence, causes and interventions","volume":"51","author":"Vrijsen","year":"2020","journal-title":"Ann. Med. Surg."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Basuita, M., Kamen, C.L., Ethier, C., and Soong, C. (2020). Reducing Inappropriate Laboratory Testing in the Hospital Setting: How Low Can We Go?. JCOM, 27.","DOI":"10.12788\/jcom.0027"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"100","DOI":"10.1016\/j.cca.2020.02.017","article-title":"Inappropriate use of laboratory tests: How availability triggers demand\u2013Examples across Europe","volume":"505","author":"Mrazek","year":"2020","journal-title":"Clin. Chim. Acta"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"e018557","DOI":"10.1136\/bmjopen-2017-018557","article-title":"Overtesting and undertesting in primary care: A systematic review and meta-analysis","volume":"8","author":"Albasri","year":"2018","journal-title":"BMJ Open"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"52","DOI":"10.1016\/j.ijmedinf.2018.05.009","article-title":"Laboratory testing in primary care: A systematic review of health IT impacts","volume":"116","author":"Maillet","year":"2018","journal-title":"Int. J. Med Inform."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"585","DOI":"10.5858\/arpa.2016-0115-RA","article-title":"The effects of computerized clinical decision support systems on laboratory test ordering: A systematic review","volume":"141","author":"Delvaux","year":"2017","journal-title":"Arch. Pathol. Lab. Med."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"253","DOI":"10.1016\/j.procs.2014.08.038","article-title":"Reducing unnecessary laboratory testing using health informatics applications: A case study on a tertiary care hospital","volume":"37","author":"Khalifa","year":"2014","journal-title":"Procedia Comput. Sci."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"994","DOI":"10.1136\/bmjqs-2014-003053","article-title":"Cost and turn-around time display decreases inpatient ordering of reference laboratory tests: A time series","volume":"23","author":"Fang","year":"2014","journal-title":"BMJ Qual. Saf."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"1833","DOI":"10.1001\/jamainternmed.2017.5152","article-title":"Evidence-based guidelines to eliminate repetitive laboratory testing","volume":"177","author":"Eaton","year":"2017","journal-title":"JAMA Intern. Med."},{"key":"ref_21","first-page":"523","article-title":"Algorithm-based decision rules to safely reduce laboratory test ordering","volume":"84","author":"Schubart","year":"2001","journal-title":"Stud. Health Technol. Inform."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"203","DOI":"10.1016\/j.cmpb.2017.03.024","article-title":"Viral warts (Human Papilloma Virus) as a potential risk for breast cancer among younger females","volume":"144","author":"Atique","year":"2017","journal-title":"Comput. Methods Programs Biomed."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"861","DOI":"10.1093\/intqhc\/mzx123","article-title":"Psychotropic medications prescribing trends in adolescents: A nationwide population-based study in Taiwan","volume":"29","author":"Tsai","year":"2017","journal-title":"Int. J. Qual. Health Care"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"1527","DOI":"10.1001\/jamainternmed.2015.3540","article-title":"Nationwide population science: Lessons from the Taiwan national health insurance research database","volume":"175","author":"Hsing","year":"2015","journal-title":"JAMA Intern. Med."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"778","DOI":"10.1093\/ajcp\/aqw064","article-title":"Using machine learning to predict laboratory test results","volume":"145","author":"Luo","year":"2016","journal-title":"Am. J. Clin. Pathol."},{"key":"ref_26","unstructured":"National Academies of Sciences, Engineering, and Medicine Improving Diagnosis in Health Care."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"1596","DOI":"10.1016\/j.cca.2010.07.022","article-title":"Managing the demand for laboratory testing: Options and opportunities","volume":"411","author":"Janssens","year":"2010","journal-title":"Clin. Chim. Acta"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"62","DOI":"10.1136\/jclinpath-2011-200524","article-title":"Managing demand for laboratory tests: A laboratory toolkit","volume":"66","author":"Fryer","year":"2013","journal-title":"J. Clin. Pathol."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"145","DOI":"10.1016\/j.cca.2013.09.023","article-title":"A clinician\u2019s perspective on laboratory utilization management","volume":"427","author":"Yeh","year":"2014","journal-title":"Clin. Chim. Acta"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1186\/s13012-020-01059-y","article-title":"Clinical decision support improves the appropriateness of laboratory test ordering in primary care without increasing diagnostic error: The ELMO cluster randomized trial","volume":"15","author":"Delvaux","year":"2020","journal-title":"Implement. Sci."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"1133","DOI":"10.5858\/arpa.2014-0034-ED","article-title":"Computational pathology: An emerging definition","volume":"138","author":"Louis","year":"2014","journal-title":"Arch. Pathol. Lab. Med."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"774","DOI":"10.1309\/AJCPX1HIEM4KLGNU","article-title":"Do we now know what inappropriate laboratory utilization is? An expanded systematic review of laboratory clinical audits","volume":"141","author":"Hauser","year":"2014","journal-title":"Am. J. Clin. Pathol."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"21","DOI":"10.1111\/j.1365-2753.2006.00641.x","article-title":"The inappropriate use of HbA1c testing to monitor glycemia: Is there evidence in laboratory data?","volume":"13","author":"Akan","year":"2007","journal-title":"J. Eval. Clin. Pract."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"784","DOI":"10.1373\/clinchem.2005.064139","article-title":"Potentially inappropriate repeat laboratory testing in inpatients","volume":"52","author":"Hawkins","year":"2006","journal-title":"Clin. Chem."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"1141","DOI":"10.5858\/2005-129-1141-ASFCQT","article-title":"A simple, focused, computerized query to detect overutilization of laboratory tests","volume":"129","author":"Weydert","year":"2005","journal-title":"Arch. Pathol. Lab. Med."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"891","DOI":"10.1016\/j.jbi.2010.09.009","article-title":"An automated technique for identifying associations between medications, laboratory results and problems","volume":"43","author":"Wright","year":"2010","journal-title":"J. Biomed. Inform."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"20","DOI":"10.1038\/538020a","article-title":"Can we open the black box of AI?","volume":"538","author":"Castelvecchi","year":"2016","journal-title":"Nat. News"},{"key":"ref_38","unstructured":"Buhrmester, V., M\u00fcnch, D., and Arens, M. (2019). Analysis of explainers of black box deep neural networks for computer vision: A survey. arXiv."}],"container-title":["Diagnostics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2075-4418\/11\/6\/990\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T06:08:56Z","timestamp":1760162936000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2075-4418\/11\/6\/990"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,5,29]]},"references-count":38,"journal-issue":{"issue":"6","published-online":{"date-parts":[[2021,6]]}},"alternative-id":["diagnostics11060990"],"URL":"https:\/\/doi.org\/10.3390\/diagnostics11060990","relation":{},"ISSN":["2075-4418"],"issn-type":[{"value":"2075-4418","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,5,29]]}}}