{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,19]],"date-time":"2026-07-19T09:50:10Z","timestamp":1784454610626,"version":"3.55.0"},"reference-count":57,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2019,2,28]],"date-time":"2019-02-28T00:00:00Z","timestamp":1551312000000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2019,2,28]],"date-time":"2019-02-28T00:00:00Z","timestamp":1551312000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["npj Digit. Med."],"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Much of the AI work in healthcare is focused around disease prediction in clinical settings, which is an important application that has yet to deliver in earnest. However, there are other fundamental aspects like helping patients and care teams interact and communicate in efficient and meaningful ways, which could deliver quadruple-aim improvements. After heart disease and cancer, preventable medical errors are the third leading cause of death in the United States. The largest subset of medical errors is medication error. Providing the right treatment plan for patients includes knowledge about their current medications and drug allergies, an often challenging task. The widespread growth of prescribing and consuming medications has increased the need for applications that support medication reconciliation. We show a deep-learning application that can help reduce avoidable errors with their attendant risk, i.e., correctly identifying prescription medication, which is currently a tedious and error-prone task. We demonstrate prescription-pill identification from mobile images in the NIH NLM Pill Image Recognition Challenge dataset. Our application recognizes the correct pill within the top-5 results at 94% accuracy, which compares favorably to the original competition winner at 83.3% for top-5 under comparable, though not identical configurations. The Institute of Medicine claims that better use of information technology can be an important step in reducing medication errors. Therefore, we believe that a more immediate impact of AI in healthcare will occur with a seamless integration of AI into clinical workflows, readily addressing the quadruple aim of healthcare.<\/jats:p>","DOI":"10.1038\/s41746-019-0086-0","type":"journal-article","created":{"date-parts":[[2019,2,28]],"date-time":"2019-02-28T11:05:29Z","timestamp":1551351929000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":53,"title":["Fast and accurate medication identification"],"prefix":"10.1038","volume":"2","author":[{"given":"Natalia","family":"Larios Delgado","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Naoto","family":"Usuyama","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Amanda K.","family":"Hall","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Rebecca J.","family":"Hazen","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Max","family":"Ma","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Siva","family":"Sahu","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jessica","family":"Lundin","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2019,2,28]]},"reference":[{"key":"86_CR1","doi-asserted-by":"publisher","first-page":"573","DOI":"10.1370\/afm.1713","volume":"12","author":"T Bodenheimer","year":"2014","unstructured":"Bodenheimer, T. & Sinsky, C. From triple to quadruple aim: care of the patient requires care of the provider. Ann. Fam. Med. 12, 573\u2013576 (2014).","journal-title":"Ann. Fam. Med."},{"key":"86_CR2","unstructured":"National Library of Medicine (NLM). Pill identification challenge. https:\/\/pir.nlm.nih.gov\/challenge\/ (2016)."},{"key":"86_CR3","unstructured":"Institute of Medicine. Preventing Medication Errors. (The National Academies Press: Washington, DC, 2007). https:\/\/www.nap.edu\/catalog\/11623."},{"key":"86_CR4","doi-asserted-by":"publisher","first-page":"i2139","DOI":"10.1136\/bmj.i2139","volume":"353","author":"MA Makary","year":"2016","unstructured":"Makary, M. A. & Daniel, M. Medical error\u2014the third leading cause of death in the us. Br. Med. J. 353, i2139 (2016).","journal-title":"Br. Med. J."},{"key":"86_CR5","doi-asserted-by":"publisher","first-page":"122","DOI":"10.1097\/PTS.0b013e3182948a69","volume":"9","author":"JT James","year":"2013","unstructured":"James, J. T. A new, evidence-based estimate of patient harms associated with hospital care. J. Patient Saf. 9, 122\u2013128 (2013).","journal-title":"J. Patient Saf."},{"key":"86_CR6","unstructured":"Institute of Medicine. To Err Is Human: Building a Safer Health System. (The National Academies Press: Washington, DC, 2000). https:\/\/www.nap.edu\/catalog\/9728."},{"key":"86_CR7","unstructured":"Agency for Healthcare Research and Quality (AHRQ). Reducing and Preventing Adverse Drug Events to Decrease Hospital Costs (2001)."},{"key":"86_CR8","doi-asserted-by":"publisher","first-page":"759","DOI":"10.1377\/hlthaff.27.3.759","volume":"27","author":"DM Berwick","year":"2008","unstructured":"Berwick, D. M., Whittington, J. & Nolan, T. W. The triple aim: care, health, and cost. Health Aff. 27, 759\u2013769 (2008).","journal-title":"Health Aff."},{"key":"86_CR9","doi-asserted-by":"publisher","first-page":"860","DOI":"10.1038\/35057062","volume":"409","author":"IHGS Consortium","year":"2001","unstructured":"Consortium, I. H. G. S. et al. Initial sequencing and analysis of the human genome. Nature 409, 860 (2001).","journal-title":"Nature"},{"key":"86_CR10","doi-asserted-by":"publisher","first-page":"1304","DOI":"10.1126\/science.1058040","volume":"291","author":"Venter JCl","year":"2001","unstructured":"JCl, Venter et al. The sequence of the human genome. Science 291, 1304\u20131351 (2001).","journal-title":"Science"},{"key":"86_CR11","doi-asserted-by":"publisher","first-page":"403","DOI":"10.1111\/j.1399-0004.2010.01609.x","volume":"79","author":"C Li","year":"2011","unstructured":"Li, C. Personalized medicine\u2014the promised land: are we there yet? Clin. Genet 79, 403\u2013412 (2011).","journal-title":"Clin. Genet"},{"key":"86_CR12","unstructured":"Mendes, E. Personalized Cancer Care: Where It Stands Today. www.cancer.org\/latest-news (2015)."},{"key":"86_CR13","unstructured":"Staff, M. C. Personalized Medicine and Pharmacogenomics. https:\/\/www.mayoclinic.org\/healthy-lifestyle\/consumer-health\/in-depth\/personalized-medicine\/art-20044300 (2018)."},{"key":"86_CR14","doi-asserted-by":"publisher","DOI":"10.1186\/s12913-017-2205-4","volume":"17","author":"A Di Paolo","year":"2017","unstructured":"Di Paolo, A., Sarkozy, F., Ryll, B. & Siebert, U. Personalized medicine in Europe: not yet personal enough? BMC Health Serv. Res. 17, 289 (2017).","journal-title":"BMC Health Serv. Res."},{"key":"86_CR15","unstructured":"National Coordinating Council for Medication Error Reporting and Prevention. What is Medication Error? https:\/\/www.nccmerp.org\/about-medication-errors (2018)."},{"key":"86_CR16","doi-asserted-by":"publisher","first-page":"424","DOI":"10.1001\/archinte.165.4.424","volume":"165","author":"PL Cornish","year":"2005","unstructured":"Cornish, P. L., Knowles, S. R., Marchesano, R., Tam, V., Shadowitz, S., Juurlink, D. N. & Etchells, E. E. Unintended medication discrepancies at the time of hospital admission. Arch. Intern. Med. 165, 424\u2013429 (2005).","journal-title":"Arch. Intern. Med."},{"key":"86_CR17","doi-asserted-by":"publisher","first-page":"18","DOI":"10.1016\/S0160-3450(15)31554-3","volume":"NS30","author":"E Larrat","year":"1990","unstructured":"Larrat, E., Taubman, A. & Willey, C. Compliance-related problems in the ambulatory population. Am. Pharm. NS30, 18\u201323 (1990).","journal-title":"Am. Pharm."},{"key":"86_CR18","unstructured":"National Library of Medicine (NLM). Pillbox. https:\/\/pillbox.nlm.nih.gov (2016)."},{"key":"86_CR19","doi-asserted-by":"crossref","unstructured":"Deeks, S. G., Lewin, S. R. & Havlir, D. V. The end of AIDS: HIV infection as a chronic disease. The Lancet 382, 1525\u20131533.","DOI":"10.1016\/S0140-6736(13)61809-7"},{"key":"86_CR20","unstructured":"Center for Drug Evaluation and Research US FDA. Advancing Health Through Innovation 2017 New Drug Therapy Approvals. https:\/\/www.fda.gov\/downloads\/AboutFDA\/CentersOffices\/OfficeofMedicalProductsandTobacco\/CDER\/ReportsBudgets\/UCM591976.pdf (2017)."},{"key":"86_CR21","unstructured":"Henry, J. Kaiser Family Foundation. Total Number of Retail Prescription Drugs Filled at Pharmacies. www.kff.org\/health-costs\/state-indicator\/total-retail-rx-drugs\/ (2018)."},{"key":"86_CR22","doi-asserted-by":"publisher","first-page":"1818","DOI":"10.1001\/jama.2015.13766","volume":"314","author":"ED Kantor","year":"2015","unstructured":"Kantor, E. D., Rehm, C. D., Haas, J. S., Chan, A. T. & Giovannucci, E. L. Trends in prescription drug use among adults in the united states from 1999\u20132012. J. Am. Med. Assoc. 314, 1818\u20131831 (2015).","journal-title":"J. Am. Med. Assoc."},{"key":"86_CR23","unstructured":"National Center for Health Statistics. Health, United States, 2016: With Chartbook on Long-Term Trends in Health. https:\/\/www.cdc.gov\/nchs\/data\/hus\/hus16.pdf (2017)."},{"key":"86_CR24","first-page":"E62","volume":"11","author":"BW Ward","year":"2014","unstructured":"Ward, B. W., Goodman, R. & Schiller, J. S. Multiple chronic conditions among us adults: a 2012 update. Prev. Chronic Dis. 11, E62 (2014).","journal-title":"Prev. Chronic Dis."},{"key":"86_CR25","doi-asserted-by":"crossref","unstructured":"Spiller, H. A. & Griffith, J. R. K. Increasing burden of pill identification requests to US poison centers. Clin. Toxicol. 47, 253\u2013255 (2009).","DOI":"10.1080\/15563650902754877"},{"key":"86_CR26","first-page":"1","volume":"56","author":"NL Hodges","year":"2017","unstructured":"Hodges, N. L., Spiller, H. A., Casavant, M. J., Chounthirath, T. & Smith, G. A. Non-health care facility medication errors resulting in serious medical outcomes. Clin. Toxicol. 56, 1\u20138 (2017).","journal-title":"Clin. Toxicol."},{"key":"86_CR27","doi-asserted-by":"crossref","unstructured":"Mowry, J. B., Spyker, D. A. Jr., Cantilena, L. R., Bailey, J. E. & Ford, M. Annual report of the american association of poison control centers\u2019 national poison data system (NPDS): 30th annual report. Clin. Toxicol. 51, 949\u20131229 (2013).","DOI":"10.3109\/15563650.2013.863906"},{"key":"86_CR28","doi-asserted-by":"publisher","first-page":"83","DOI":"10.1056\/NEJMhle1101722","volume":"365","author":"AJ Greene","year":"2011","unstructured":"Greene, A. J. & Kesselheim, A. S. Why do the same drugs look different? pills, trade dress, and public health. New Engl. J. Med. 365, 83\u201389 (2011).","journal-title":"New Engl. J. Med."},{"key":"86_CR29","unstructured":"Zeng, X., Cao, K. & Zhang, M. MobileDeepPill: A small-footprint mobile deep learning system for recognizing unconstrained pill images. In Proc. of the 15th ACM Int. Conference on Mobile Systems, Applications, and Services (MobiSys), Cuervo, E. (ed.) 56\u201367 (ACM, Niagara Falls, NY, USA, 2017)."},{"key":"86_CR30","unstructured":"Dalal, N. & Triggs, B. Histograms of oriented gradients for human detection. In Proc. of the IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), Schmid, C. et al. (eds.), 886\u2013893 (IEEE, San Diego, CA, USA, 2005)."},{"key":"86_CR31","doi-asserted-by":"crossref","unstructured":"Wang, J. et al. Learning fine-grained image similarity with deep ranking. In Proc. of the 2014 IEEE Conf. on Computer Vision and Pattern Recognition, Mortensen, E. & Fidler, S. (eds.), 1386\u20131393 (IEEE, Columbus, OH, USA, 2014).","DOI":"10.1109\/CVPR.2014.180"},{"key":"86_CR32","doi-asserted-by":"crossref","unstructured":"Wang, Y., Ribera, J., Liu, C., Yarlagadda, S. & Zhu, F. Pill recognition using minimal labeled data. in Proc. of the IEEE Third International Conf. on Multimedia Big Data (BigMM), Kim, M. (ed.), 346\u2013353 (IEEE, Laguna Hills, CA, USA, 2017).","DOI":"10.1109\/BigMM.2017.61"},{"key":"86_CR33","doi-asserted-by":"crossref","unstructured":"Szegedy, C. et al. Going Deeper With Convolutions (Cvpr 2015).","DOI":"10.1109\/CVPR.2015.7298594"},{"key":"86_CR34","doi-asserted-by":"crossref","unstructured":"Canny, J. A computational approach to edge detection. IEEE Trans. Pattern Anal. Mach. Intell. 8, 679\u2013698 (1987).","DOI":"10.1109\/TPAMI.1986.4767851"},{"key":"86_CR35","doi-asserted-by":"publisher","first-page":"130","DOI":"10.1016\/j.jbi.2017.09.005","volume":"74","author":"YF Wong","year":"2017","unstructured":"Wong, Y. F. et al. Development of fine-grained pill identification algorithm using deep convolutional network. J. Biomed. Inform. 74, 130\u2013136 (2017).","journal-title":"J. Biomed. Inform."},{"key":"86_CR36","unstructured":"Krizhevsky, A., Sutskever, I. & Hinton, G. E. ImageNet classification with deep convolutional neural networks. In Advances in Neural Information Processing Systems, Pereira, F. et al. (eds.), 1097\u20131105 (Curran Assoc., Lake Tahoe, NV, USA, 2012)."},{"key":"86_CR37","doi-asserted-by":"publisher","first-page":"904","DOI":"10.1016\/j.patrec.2011.08.022","volume":"33","author":"YB Lee","year":"2012","unstructured":"Lee, Y.-B., Park, U., Jain, A. K. & Lee, S.-W. Pill-id: matching and retrieval of drug pill images. Pattern Recognit. Lett. 33, 904\u2013910 (2012).","journal-title":"Pattern Recognit. Lett."},{"key":"86_CR38","first-page":"179","volume":"8","author":"MK Hu","year":"1962","unstructured":"Hu, M.-K. Visual pattern recognition by moment invariants. IRE Trans. Inf. Theory 8, 179\u2013187 (1962).","journal-title":"IRE Trans. Inf. Theory"},{"key":"86_CR39","doi-asserted-by":"publisher","first-page":"1523","DOI":"10.1016\/j.protcy.2014.10.174","volume":"16","author":"A Cunha","year":"2014","unstructured":"Cunha, A., Ad\u00e3o, T. & Trigueiros, P. Helpmepills: a mobile pill recognition tool for elderly persons. Proc. Technol. 16, 1523\u20131532 (2014).","journal-title":"Proc. Technol."},{"key":"86_CR40","doi-asserted-by":"crossref","unstructured":"Caban, J. J., Rosebrock, A. & Yoo, T. S. Automatic identification of prescription drugs using shape distribution models. In 2012 19th IEEE International Conference on Image Processing (ICIP), Rao, R. & Gurram, P. (eds.), 1005\u20131008 (IEEE, Orlando, FL, USA, 2012).","DOI":"10.1109\/ICIP.2012.6467032"},{"key":"86_CR41","doi-asserted-by":"crossref","unstructured":"Yu, J., Chen, Z. & Kamata, S. Pill recognition using imprint information by two-step sampling distance sets. In 2014 22nd International Conference on Pattern Recognition (ICPR), Felsberg, M. (ed.), 3156\u20133161 (IEEE, Stockholm, 2014).","DOI":"10.1109\/ICPR.2014.544"},{"key":"86_CR42","doi-asserted-by":"crossref","unstructured":"Yu, J., Chen, Z., Kamata, S. & Yang, J. Accurate system for automatic pill recognition using imprint information. IET Image Process. 9, 1039\u20131047 (2015).","DOI":"10.1049\/iet-ipr.2014.1007"},{"key":"86_CR43","doi-asserted-by":"publisher","first-page":"5","DOI":"10.1023\/A:1010933404324","volume":"45","author":"L Breiman","year":"2001","unstructured":"Breiman, L. Random forests. Mach. Learn. 45, 5\u201332 (2001).","journal-title":"Mach. Learn."},{"key":"86_CR44","first-page":"2579","volume":"9","author":"Lvd Maaten","year":"2008","unstructured":"Maaten, Lvd & Hinton, G. Visualizing data using t-sne. J. Mach. Learn. Res. 9, 2579\u20132605 (2008).","journal-title":"J. Mach. Learn. Res."},{"key":"86_CR45","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S. & Sun, J. Deep Residual learning for image recognition. In Procs of the IEEE Conf. on Computer Vision and Pattern Recognition, Mortensen, E. & Saenko, K. (eds.), 770\u2013778 (IEEE, Las Vegas, NV, USA, 2016).","DOI":"10.1109\/CVPR.2016.90"},{"key":"86_CR46","unstructured":"Choromanska, A., Agarwal, A. & Langford, J. Extreme multi-class classification. In NIPS Workshop on Multi-Class and Multi-Label Learning with Millions of Categories, Varma, M. (ed.), vol. 1, 1\u20132 (NIPS, Lake Tahoe, NV, USA, 2013)."},{"key":"86_CR47","doi-asserted-by":"crossref","unstructured":"Oh Song, H., Xiang, Y., Jegelka, S. & Savarese, S. Deep metric learning via lifted structured feature embedding. In Procs of the IEEE Conf. on Computer Vision and Pattern Recognition, Mortensen E. & Saenko K. (eds.), 4004\u20134012 (IEEE, Las Vegas, NV, USA, 2016).","DOI":"10.1109\/CVPR.2016.434"},{"key":"86_CR48","doi-asserted-by":"crossref","unstructured":"Long, J., Shelhamer, E. & Darrell, T. Fully convolutional networks for semantic segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 39, 640\u2013651 (2017).","DOI":"10.1109\/TPAMI.2016.2572683"},{"key":"86_CR49","doi-asserted-by":"publisher","first-page":"211","DOI":"10.1007\/s11263-015-0816-y","volume":"115","author":"O Russakovsky","year":"2015","unstructured":"Russakovsky, O. et al. Imagenet large scale visual recognition challenge. Int. J. Comput. Vision. 115, 211\u2013252 (2015).","journal-title":"Int. J. Comput. Vision."},{"key":"86_CR50","doi-asserted-by":"crossref","unstructured":"Ronneberger, O., Fischer, P. & Brox, T. U-net: convolutional networks for biomedical image segmentation. In Int. Conf. on Medical Image Computing and Computer-Assisted Intervention (MICCAI), Navab N., Hornegger J., Wells, W. & Frangi, A. (eds.), 234\u2013241 (Springer, Munich, 2015).","DOI":"10.1007\/978-3-319-24574-4_28"},{"key":"86_CR51","unstructured":"Kingma, D. P. & Ba, J. Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)."},{"key":"86_CR52","unstructured":"Iandola, F. N. et al. SqueezeNet: AlexNet-level accuracy with 50\u00d7 fewer parameters and <0.5MB model size. ArXiv e-prints (2016)."},{"key":"86_CR53","unstructured":"Howard, A. G. et al. MobileNets: efficient convolutional neural networks for mobile vision applications. ArXiv e-prints (2017)."},{"key":"86_CR54","doi-asserted-by":"crossref","unstructured":"Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J. & Wojna, Z. Rethinking the Inception Architecture for Computer Vision. CoRR abs\/1512.00567 (2015).","DOI":"10.1109\/CVPR.2016.308"},{"key":"86_CR55","unstructured":"Abadi, M. et al. TensorFlow: Large-Scale Machine Learning on Heterogeneous Systems. https:\/\/www.tensorflow.org\/ (2015)."},{"key":"86_CR56","doi-asserted-by":"crossref","unstructured":"Editorial. Combating counterfeit drugs. The Lancet 371 (2008).","DOI":"10.1016\/S0140-6736(08)60663-7"},{"key":"86_CR57","unstructured":"Chollet, F. et al. Keras. https:\/\/keras.io (2015)."}],"container-title":["npj Digital Medicine"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.nature.com\/articles\/s41746-019-0086-0.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s41746-019-0086-0","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s41746-019-0086-0.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,12,17]],"date-time":"2022-12-17T18:27:55Z","timestamp":1671301675000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.nature.com\/articles\/s41746-019-0086-0"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,2,28]]},"references-count":57,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2019,12]]}},"alternative-id":["86"],"URL":"https:\/\/doi.org\/10.1038\/s41746-019-0086-0","relation":{},"ISSN":["2398-6352"],"issn-type":[{"value":"2398-6352","type":"electronic"}],"subject":[],"published":{"date-parts":[[2019,2,28]]},"assertion":[{"value":"12 June 2018","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"7 February 2019","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"28 February 2019","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"All the authors are employees from the Microsoft Corporation and this research work covers methods and models that might be used in a Microsoft service in the future. This could create an appearance of benefit.","order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"10"}}