{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,12]],"date-time":"2026-05-12T18:52:17Z","timestamp":1778611937701,"version":"3.51.4"},"reference-count":18,"publisher":"Oxford University Press (OUP)","issue":"2","license":[{"start":{"date-parts":[[2023,10,9]],"date-time":"2023-10-09T00:00:00Z","timestamp":1696809600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/academic.oup.com\/pages\/standard-publication-reuse-rights"}],"funder":[{"name":"National Library Medicine","award":["R01 LM013337"],"award-info":[{"award-number":["R01 LM013337"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2024,1,18]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:sec>\n                  <jats:title>Objective<\/jats:title>\n                  <jats:p>Reflex testing protocols allow clinical laboratories to perform second line diagnostic tests on existing specimens based on the results of initially ordered tests. Reflex testing can support optimal clinical laboratory test ordering and diagnosis. In current clinical practice, reflex testing typically relies on simple \u201cif-then\u201d rules; however, this limits the opportunities for reflex testing since most test ordering decisions involve more complexity than traditional rule-based approaches would allow. Here, using the analyte ferritin as an example, we propose an alternative machine learning-based approach to \u201csmart\u201d reflex testing.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Methods<\/jats:title>\n                  <jats:p>Using deidentified patient data, we developed a machine learning model to predict whether a patient getting CBC testing will also have ferritin testing ordered. We evaluate applications of this model to reflex testing by assessing its performance in comparison to possible rule-based approaches.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Results<\/jats:title>\n                  <jats:p>Our underlying machine learning models performed moderately well in predicting ferritin test ordering (AUC=0.731 in reference to actual ordering) and demonstrated promising potential to underlie key clinical applications. In contrast, none of the many traditionally framed, rule-based, hypothetical reflex protocols we evaluated offered sufficient agreement with actual ordering to be clinically feasible. Using chart review, we further demonstrated that the strategic deployment of our model could avoid important ferritin test ordering errors.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Conclusions<\/jats:title>\n                  <jats:p>Machine learning may provide a foundation for new types of reflex testing with enhanced benefits for clinical diagnosis.<\/jats:p>\n               <\/jats:sec>","DOI":"10.1093\/jamia\/ocad187","type":"journal-article","created":{"date-parts":[[2023,10,9]],"date-time":"2023-10-09T20:59:05Z","timestamp":1696885145000},"page":"416-425","source":"Crossref","is-referenced-by-count":6,"title":["Using machine learning to develop smart reflex testing protocols"],"prefix":"10.1093","volume":"31","author":[{"given":"Matthew","family":"McDermott","sequence":"first","affiliation":[{"name":"MIT Computer Science and Artificial Intelligence Lab , Boston, MA 02139, United States"},{"name":"Department of Biomedical Informatics, Harvard Medical School , Boston, MA 02115, United States"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Anand","family":"Dighe","sequence":"additional","affiliation":[{"name":"Department of Pathology, Massachusetts General Hospital , Boston, MA 02114, United States"},{"name":"Harvard Medical School , Boston, MA, United States"},{"name":"MGB HealthCare System , Somerville, MA 02145, United States"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Peter","family":"Szolovits","sequence":"additional","affiliation":[{"name":"MIT Computer Science and Artificial Intelligence Lab , Boston, MA 02139, United States"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0195-7456","authenticated-orcid":false,"given":"Yuan","family":"Luo","sequence":"additional","affiliation":[{"name":"Department of Preventive Medicine, Northwestern University , Chicago, IL 60611, United States"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4182-6349","authenticated-orcid":false,"given":"Jason","family":"Baron","sequence":"additional","affiliation":[{"name":"Department of Pathology, Massachusetts General Hospital , Boston, MA 02114, United States"},{"name":"Harvard Medical School , Boston, MA, United States"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"286","published-online":{"date-parts":[[2023,10,9]]},"reference":[{"key":"2024011907291699000_ocad187-B1","doi-asserted-by":"crossref","first-page":"111","DOI":"10.1016\/j.cca.2013.09.021","article-title":"Utilization management in the clinical laboratory: an introduction and overview of the literature","volume":"427","author":"Huck","year":"2014","journal-title":"Clin Chim Acta"},{"issue":"10270","key":"2024011907291699000_ocad187-B2","doi-asserted-by":"crossref","first-page":"233","DOI":"10.1016\/S0140-6736(20)32594-0","article-title":"Iron deficiency","volume":"397","author":"Pasricha","year":"2021","journal-title":"Lancet"},{"issue":"6","key":"2024011907291699000_ocad187-B3","doi-asserted-by":"crossref","first-page":"567","DOI":"10.1056\/NEJMcpc1503829","article-title":"Case records of the Massachusetts General Hospital. Case 24-2016. A 66-year-old man with malaise, weakness, and hypercalcemia","volume":"375","author":"Bazari","year":"2016","journal-title":"N Engl J Med"},{"key":"2024011907291699000_ocad187-B4","doi-asserted-by":"crossref","first-page":"196","DOI":"10.1016\/j.cca.2013.09.027","article-title":"The role of informatics and decision support in utilization management","volume":"427","author":"Baron","year":"2014","journal-title":"Clin Chim Acta"},{"key":"2024011907291699000_ocad187-B5","doi-asserted-by":"crossref","first-page":"81","DOI":"10.1007\/978-3-319-34199-6","volume-title":"Utilization Management in the Clinical Laboratory and Other Ancillary Services","author":"Lewandrowski","year":"2017","edition":"1st ed"},{"key":"2024011907291699000_ocad187-B6","doi-asserted-by":"crossref","first-page":"35","DOI":"10.4103\/2153-3539.83740","article-title":"Computerized provider order entry in the clinical laboratory","volume":"2","author":"Baron","year":"2011","journal-title":"J Pathol Inform"},{"issue":"7","key":"2024011907291699000_ocad187-B7","doi-asserted-by":"crossref","first-page":"994","DOI":"10.1373\/clinchem.2017.280214","article-title":"What\u2019s new in laboratory test utilization management?","volume":"64","author":"Baird","year":"2018","journal-title":"Clin Chem"},{"issue":"2","key":"2024011907291699000_ocad187-B8","doi-asserted-by":"crossref","first-page":"145","DOI":"10.1007\/BF02598003","article-title":"Laboratory diagnosis of iron-deficiency anemia: an overview","volume":"7","author":"Guyatt","year":"1992","journal-title":"J Gen Intern Med"},{"key":"2024011907291699000_ocad187-B9","first-page":"2825","article-title":"Scikit-learn: machine learning in Python","volume":"12","author":"Pedregosa","year":"2011","journal-title":"J Mach Learn Res"},{"key":"2024011907291699000_ocad187-B10","volume-title":"R: A Language and Environment for Statistical Computing","author":"R Core Team","year":"2013"},{"key":"2024011907291699000_ocad187-B11","doi-asserted-by":"crossref","DOI":"10.1007\/978-3-319-24277-4","volume-title":"ggplot2: Elegant Graphics for Data Analysis","author":"Wickham","year":"2016"},{"issue":"1","key":"2024011907291699000_ocad187-B12","doi-asserted-by":"crossref","first-page":"ix","DOI":"10.1016\/j.cll.2022.09.001","article-title":"Artificial intelligence in the clinical laboratory","volume":"43","author":"Baron","year":"2023","journal-title":"Clin Lab Med"},{"key":"2024011907291699000_ocad187-B13","author":"Cao","year":"2018"},{"issue":"1","key":"2024011907291699000_ocad187-B14","doi-asserted-by":"crossref","first-page":"bbab489","DOI":"10.1093\/bib\/bbab489","article-title":"Evaluating the state of the art in missing data imputation for clinical data","volume":"23","author":"Luo","year":"2022","journal-title":"Brief Bioinform"},{"key":"2024011907291699000_ocad187-B15","first-page":"435","author":"Thakur","year":"2021"},{"issue":"6","key":"2024011907291699000_ocad187-B16","doi-asserted-by":"crossref","first-page":"645","DOI":"10.1093\/jamia\/ocx133","article-title":"3D-MICE: integration of cross-sectional and longitudinal imputation for multi-analyte longitudinal clinical data","volume":"25","author":"Luo","year":"2018","journal-title":"J Am Med Inform Assoc"},{"key":"2024011907291699000_ocad187-B17","doi-asserted-by":"crossref","first-page":"21689","DOI":"10.1038\/srep21689","article-title":"Multiple imputation for general missing data patterns in the presence of high-dimensional data","volume":"6","author":"Deng","year":"2016","journal-title":"Sci Rep"},{"key":"2024011907291699000_ocad187-B18","doi-asserted-by":"crossref","first-page":"84","DOI":"10.1016\/j.jbi.2013.12.005","article-title":"Decision support from local data: creating adaptive order menus from past clinician behavior","volume":"48","author":"Klann","year":"2014","journal-title":"J Biomed Inform"}],"container-title":["Journal of the American Medical Informatics Association"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/jamia\/article-pdf\/31\/2\/416\/56209448\/ocad187.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/jamia\/article-pdf\/31\/2\/416\/56209448\/ocad187.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,1,19]],"date-time":"2024-01-19T07:32:17Z","timestamp":1705649537000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/jamia\/article\/31\/2\/416\/7301309"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,10,9]]},"references-count":18,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2023,10,9]]},"published-print":{"date-parts":[[2024,1,18]]}},"URL":"https:\/\/doi.org\/10.1093\/jamia\/ocad187","relation":{},"ISSN":["1067-5027","1527-974X"],"issn-type":[{"value":"1067-5027","type":"print"},{"value":"1527-974X","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2024,2,1]]},"published":{"date-parts":[[2023,10,9]]}}}